feat: add S3 import functionality which is supported for MySQL instan…

…ces (terraform-aws-modules#289)

README.md

@@ -139,7 +139,7 @@ module "db" {
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers
@@ -214,9 +214,10 @@ No resources.
 | performance\_insights\_retention\_period | The amount of time in days to retain Performance Insights data. Either 7 (7 days) or 731 (2 years). | `number` | `7` | no |
 | port | The port on which the DB accepts connections | `string` | n/a | yes |
 | publicly\_accessible | Bool to control if instance is publicly accessible | `bool` | `false` | no |
-| replicate\_source\_db | Specifies that this resource is a Replicate database, and to use this value as the source database. This correlates to the identifier of another Amazon RDS Database to replicate. | `string` | `""` | no |
+| replicate\_source\_db | Specifies that this resource is a Replicate database, and to use this value as the source database. This correlates to the identifier of another Amazon RDS Database to replicate. | `string` | `null` | no |
+| s3\_import | Restore from a Percona Xtrabackup in S3 (only MySQL is supported) | `map(string)` | `null` | no |
 | skip\_final\_snapshot | Determines whether a final DB snapshot is created before the DB instance is deleted. If true is specified, no DBSnapshot is created. If false is specified, a DB snapshot is created before the DB instance is deleted, using the value from final\_snapshot\_identifier | `bool` | `true` | no |
-| snapshot\_identifier | Specifies whether or not to create this database from a snapshot. This correlates to the snapshot ID you'd find in the RDS console, e.g: rds:production-2015-06-26-06-05. | `string` | `""` | no |
+| snapshot\_identifier | Specifies whether or not to create this database from a snapshot. This correlates to the snapshot ID you'd find in the RDS console, e.g: rds:production-2015-06-26-06-05. | `string` | `null` | no |
 | storage\_encrypted | Specifies whether the DB instance is encrypted | `bool` | `false` | no |
 | storage\_type | One of 'standard' (magnetic), 'gp2' (general purpose SSD), or 'io1' (provisioned IOPS SSD). The default is 'io1' if iops is specified, 'gp2' if not. | `string` | `"gp2"` | no |
 | subnet\_ids | A list of VPC subnet IDs | `list(string)` | `[]` | no |

examples/complete-mssql/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/complete-mssql/outputs.tf

@@ -51,6 +51,7 @@ output "this_db_instance_username" {
 output "this_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.db.this_db_instance_password
+  sensitive   = true
 }
 
 output "this_db_instance_port" {

examples/complete-mssql/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/complete-mysql/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/complete-mysql/outputs.tf

@@ -51,6 +51,7 @@ output "this_db_instance_username" {
 output "this_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.db.this_db_instance_password
+  sensitive   = true
 }
 
 output "this_db_instance_port" {

examples/complete-mysql/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/complete-oracle/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/complete-oracle/outputs.tf

@@ -51,6 +51,7 @@ output "this_db_instance_username" {
 output "this_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.db.this_db_instance_password
+  sensitive   = true
 }
 
 output "this_db_instance_port" {

examples/complete-oracle/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/complete-postgres/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/complete-postgres/outputs.tf

@@ -51,6 +51,7 @@ output "this_db_instance_username" {
 output "this_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.db.this_db_instance_password
+  sensitive   = true
 }
 
 output "this_db_instance_port" {

examples/complete-postgres/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/enhanced-monitoring/README.md

@@ -23,7 +23,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/enhanced-monitoring/outputs.tf

@@ -51,6 +51,7 @@ output "this_db_instance_username" {
 output "this_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.db.this_db_instance_password
+  sensitive   = true
 }
 
 output "this_db_instance_port" {

examples/enhanced-monitoring/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/replica-mysql/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/replica-mysql/outputs.tf

@@ -52,6 +52,7 @@ output "master_db_instance_username" {
 output "master_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.master.this_db_instance_password
+  sensitive   = true
 }
 
 output "master_db_instance_port" {

examples/replica-mysql/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/replica-postgres/README.md

@@ -21,7 +21,7 @@ Note that this example may create resources which cost money. Run `terraform des
 
 | Name | Version |
 |------|---------|
-| terraform | >= 0.12.6 |
+| terraform | >= 0.12.26 |
 | aws | >= 2.49 |
 
 ## Providers

examples/replica-postgres/outputs.tf

@@ -52,6 +52,7 @@ output "master_db_instance_username" {
 output "master_db_instance_password" {
   description = "The database password (this password may be old, because Terraform doesn't track it after initial creation)"
   value       = module.master.this_db_instance_password
+  sensitive   = true
 }
 
 output "master_db_instance_port" {

examples/replica-postgres/versions.tf

@@ -1,7 +1,10 @@
 terraform {
-  required_version = ">= 0.12.6"
+  required_version = ">= 0.12.26"
 
   required_providers {
-    aws = ">= 2.49"
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.49"
+    }
   }
 }

examples/s3-import-mysql/README.md

@@ -0,0 +1,104 @@
+# MySQL S3 Import Example
+
+Configuration in this directory creates set of RDS resources including DB instance, DB subnet group and DB parameter group where the database itself is imported from a MySQL Percona Xtrabackup stored in S3.
+
+## Usage
+
+To run this example you need to execute:
+
+```bash
+$ terraform init
+$ terraform plan
+$ terraform apply
+```
+
+## Testing
+
+In order to test this example, you will need a database backup in order to upload to S3 and import into the module. A backup has been provided under `backup/`, but in the case that a new backup needs to be created, the steps outlined below should suffice for creating a backup that can be used for the sake of testing and verifying module functionality/changes.
+
+1. Create database container
+
+```bash
+$ docker run -d --name percona-server-mysql-8.0.20 -e MYSQL_ROOT_PASSWORD=root percona/percona-server:8.0.20
+$ docker exec -it percona-server-mysql-8.0.20 bash
+$ mysql -u root -p # password is also root
+```
+
+2. Once logged into container and database, create database and user used by RDS
+
+```sql
+CREATE DATABASE s3Import;
+CREATE USER 's3_import_user'@'localhost' IDENTIFIED BY 'YourPwdShouldBeLongAndSecure!';
+GRANT ALL PRIVILEGES ON * . * TO 's3_import_user'@'localhost';
+FLUSH PRIVILEGES;
+```
+
+3. Use Percona Xtrabackup container to dump database and upload to S3
+
+```bash
+$ mkdir -p /tmp/backup
+$ docker run --name percona-xtrabackup-8.0 --mount type=bind,src=/tmp/backup,dst=/backup --volumes-from percona-server-mysql-8.0.20 percona/percona-xtrabackup:8.0 xtrabackup --backup --data-dir=/var/lib/mysql --target-dir=/backup --user=root --password=root
+$ mv /tmp/backup ./backup
+```
+
+Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources.
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| terraform | >= 0.12.26 |
+| aws | >= 2.49 |
+| random | >= 3.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| aws | >= 2.49 |
+| random | >= 3.0 |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| db | ../../ |  |
+| import_s3_bucket | terraform-aws-modules/s3-bucket/aws | 1.17.0 |
+| security_group | terraform-aws-modules/security-group/aws | ~> 3.17 |
+| vpc | terraform-aws-modules/vpc/aws | 2.70.0 |
+
+## Resources
+
+| Name |
+|------|
+| [aws_iam_policy_document](https://registry.terraform.io/providers/hashicorp/aws/2.49/docs/data-sources/iam_policy_document) |
+| [aws_iam_role_policy](https://registry.terraform.io/providers/hashicorp/aws/2.49/docs/resources/iam_role_policy) |
+| [aws_iam_role](https://registry.terraform.io/providers/hashicorp/aws/2.49/docs/resources/iam_role) |
+| [random_pet](https://registry.terraform.io/providers/hashicorp/random/3.0/docs/resources/pet) |
+
+## Inputs
+
+No input.
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| this\_db\_instance\_address | The address of the RDS instance |
+| this\_db\_instance\_arn | The ARN of the RDS instance |
+| this\_db\_instance\_availability\_zone | The availability zone of the RDS instance |
+| this\_db\_instance\_endpoint | The connection endpoint |
+| this\_db\_instance\_hosted\_zone\_id | The canonical hosted zone ID of the DB instance (to be used in a Route 53 Alias record) |
+| this\_db\_instance\_id | The RDS instance ID |
+| this\_db\_instance\_name | The database name |
+| this\_db\_instance\_password | The database password (this password may be old, because Terraform doesn't track it after initial creation) |
+| this\_db\_instance\_port | The database port |
+| this\_db\_instance\_resource\_id | The RDS Resource ID of this instance |
+| this\_db\_instance\_status | The RDS instance status |
+| this\_db\_instance\_username | The master username for the database |
+| this\_db\_parameter\_group\_arn | The ARN of the db parameter group |
+| this\_db\_parameter\_group\_id | The db parameter group id |
+| this\_db\_subnet\_group\_arn | The ARN of the db subnet group |
+| this\_db\_subnet\_group\_id | The db subnet group name |
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->