Merge pull request Azure#657 from Azure/bastion_update

Update Bastion with no public IP Linux and Windows VMs

blocks/keyvault-secret.json

@@ -0,0 +1,33 @@
+{
+  "location": "variables.location",
+  "resource_group": "variables.resource_group",
+  "admin_user": "variables.admin_user",
+  "variables": {
+    "location": "<NOT-SET>",
+    "resource_group": "<NOT-SET>",
+    "key_vault": "<NOT-SET>",
+    "secret_name": "<NOT-SET>",
+    "admin_user": "hpcadmin"
+  },
+  "vnet": {
+  },
+  "resources": {},
+  "install": [
+    {
+      "type": "local_script",
+      "script": "create_keyvault.sh",
+      "args": [
+        "variables.resource_group",
+        "variables.key_vault"
+      ]
+    },
+    {
+      "type": "local_script",
+      "script": "create_password_in_keyvault.sh",
+      "args": [
+        "variables.key_vault",
+        "variables.secret_name"
+      ]
+    }
+  ]
+}

examples/bastion/bastion_ssh_jumpbox.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+set -euo pipefail
+
+CONFIG_FILE=${1:-config.json}
+
+ResourceGroup=$(grep resource_group ${CONFIG_FILE} | grep -v variables | awk -F'"' '{print $4}')
+User=$(grep admin_user ${CONFIG_FILE} | grep -v variables | awk -F'"' '{print $4}')
+JumpboxName=bastion-jumpbox
+
+BastionName=$(az network bastion list -g ${ResourceGroup} --query '[].name' --output tsv)
+TargetResourceId=$(az vm show -g ${ResourceGroup} --name ${JumpboxName} --query 'id' --output tsv)
+SshPrivateKey=${User}_id_rsa
+
+az network bastion ssh --name $BastionName \
+                       --resource-group $ResourceGroup \
+                       --target-resource-id $TargetResourceId \
+                       --auth-type "ssh-key" \
+                       --username $User \
+                       --ssh-key $SshPrivateKey

examples/bastion/init.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# Ensure that jq is installed
+command -v jq &> /dev/null || { echo -e >&2 "ERROR: Missing requirement: jq\nMake sure it is installed and its installation path included in PATH before executing $0"; exit 1; }
+
+# Find Linux distro from image name
+os_image=$(jq -r '.variables.jumpbox_image' variables.json)
+shopt -s nocasematch
+if [[ "$os_image" =~ "centos" ]]; then
+    os_type=centos
+elif [[ "$os_image" =~ "ubuntu" ]]; then
+    os_type=ubuntu
+else
+    echo "ERROR: Unsupported Linux distribution of image: $os_image"
+    exit 1
+fi
+
+# Create prerequisites configuration file
+echo "{}" > prereqs.json
+prereqs="$azhpc_dir/blocks/keyvault-secret.json"
+$azhpc_dir/init-and-merge.sh $prereqs prereqs.json variables.json
+
+# Create config file
+echo "{}" > config.json
+$azhpc_dir/init-and-merge.sh ./templates/config-template.json config.json variables.json
+ci_file="@templates/cloud-init-${os_type}.txt"
+jq --arg cif "$ci_file" '.variables.customdata = $cif' config.json > config.tmp && mv config.tmp config.json
+
+# Just a little cleanup...
+rm -f keyvault-secret.json config-template.json

examples/bastion/readme.md

@@ -1,66 +1,100 @@
-# Build a compute cluster with no public IP access, log-on using Azure Bastion
+# Deploy Azure Bastion for SSH and RDP connections to dedicated jumpbox VMs
 
-Visualisation: [config.json](https://azurehpc.azureedge.net/o=https://raw.githubusercontent.com/Azure/azurehpc/master/examples/bastion/config.json)
+This example will create a Bastion service to connect to a Linux jumpbox via SSH and a Windows VM via RDP. All VMs are configured without public IP for maximum security.
+Additionally, the following components are installed in the Linux jumpbox using a cloud-init script:
+* git
+* jq
+* AzureHPC
+* azcopy
+* azcli
 
-This example will create an HPC cluster wth no public IP, you can log-in using Azure Bastion, from the Portal RDP to a Windows VM or ssh to a linux VM.
->Note: The config_no_pub_ip.json deploys an Azure Bastion, VNET and a jumpbox (no pub IP), then you can login to the jumpbox via the azure bastion and deploy the rest of your azurehpc deployment. The config_no_pub_ip.json contains an example of using cloud-init in AzureHPC. The AzureHPC prerequisites are installed on the jumpbox (with no public IP) using a cloud-init script (cloud-init.txt (Installs git, jq, AzureHPC git clone, azcopy and azcli). The Cloud-init script needs to be either uploaded to blob storage in advance or referenced in the config file by @cloud-init.txt (read-in directly).
+The Linux jumpbox supports CentOS and Ubuntu images.
 
-## Initialise the project
+## Step 1 - Install and initialize AzureHPC
 
-To start you need to copy this directory and update the `config.json`.  Azurehpc provides the `azhpc-init` command that can help here by compying the directory and substituting the unset variables.  First run with the `-s` parameter to see which variables need to be set:
+Clone the `azhpc` repository and source the `install.sh` script.
 
 ```
-azhpc-init -c $azhpc_dir/examples/bastion -d bastion -s
+git clone https://github.com/Azure/azurehpc.git
+source azurehpc/install.sh
 ```
 
-The variables can be set with the `-v` option where variables are comma separated.  The output from the previous command as a starting point.  The `-d` option is required and will create a new directory name for you.  Please update to whatever `resource_group` you would like to deploy to:
+## Step 2 - Initialize the project
+
+To start you need to copy this directory in the desired working location and update the `variables.json` file with the desired parameters.
+
+| Variable                     | Value                                                                   |
+|------------------------------|-------------------------------------------------------------------------|
+| **resource_group**           | The resource group to put the resources                                 |
+| **location**                 | Azure region to deploy resources                                        |
+| **vnet_ip_range**            | IP address range in CIDR notation for Bastion VNet                      |
+| **default_subnet_ip_range**  | IP address range in CIDR notation for VMs subnet                        |
+| **bastion_subnet_ip_range**  | IP address range in CIDR notation for Bastion subnet                    |
+| **jumpbox_image**            | CentOS or Ubuntu marketplace image for Linux jumpbox                    |
+| **key_vault**                | Unique name to assign to Key Vault                                      |
+| **secret_name**              | **DO NOT MODIFY** - Name of the secret storing Windows VM user password |
+
+Then run the `init.sh` script to automatically create the `prereqs.json` and `config.json` configuration files:
 
 ```
-azhpc-init -c $azhpc_dir/examples/bastion -d bastion -v resource_group=azurehpc-cluster
+./init.sh
 ```
 
-> Note:  You can still update variables even if they are already set.  For example, in the command below we change the region to `westus2` and the SKU to `Standard_HC44rs`:
+## Step 3 - Create the Key Vault and secret
+
+Before deploying the VMs, a Key Vault must be created containing the future Windows VM password as secret.
+This is done by AzureHPC through the `prereqs.json` configuration file. Here is the command:
 
 ```
-azhpc-init -c $azhpc_dir/examples/bastion -d bastion -v location=westus2,vm_type=Standard_HC44rs,resource_group=azhpc-cluster
+azhpc-build --no-vnet -c prereqs.json
 ```
 
-## Create the cluster with no public IP
+## Step 4 - Create Bastion and jumpbox VMs
+
+To start the Bastion and jumpbox VMs deployment execute the following command:
 
 ```
-cd bastion
 azhpc-build
 ```
 
-Allow ~15 minutes for deployment.  You are able to view the status VMs being deployed by running `azhpc-status` in another terminal.
+Allow about 15 minutes to completion.
 
-## Log in the cluster
+## Step 5 - Access the Linux jumpbox via SSH
 
-Connect to the linux headnode using Azure bastion service via ssh
+### From local shell terminal
 
-You can login to a VM via bastion directly from your linux workstation using
+The provided `bastion_ssh_jumpbox.sh` script allows to easily access the jumpbox VM via SSH.
+
+Simply run the script to log into the jumpbox VM:
 
 ```
-bastion_ssh_bjumpbox.sh
+./bastion_ssh_jumpbox.sh
 ```
->Note: You will need to edit this script to provide the BastionName, ResourceGroup, TargetResourceId, User and SshPrivateKey
 
-You can also use the Azure portal to login to you VM via bastion, see below.
+### From Azure Portal
+
+You can also use the Azure Portal to login to the jumpbox VM via Bastion.
+
+Locate the `bastion-jumpbox` VM on the Azure portal and click on "Connect" menu button. Select the "Bastion" option.
+
+![Alt text](/examples/bastion/images/jumpbox_connect.png?raw=true "Jumpbox Connect menu button")
 
-Locate the VM you want to connect to on the Azure portal and check "Connect".
+In the Bastion pane type `hpcadmin` in the "Username" field and select "SSH Private Key from Local File" to provide the `hpcadmin_id_rsa` private key created by AzureHPC in the directory where `azhpc-build` has been executed.
 
-![Alt text](/examples/bastion/images/bastion_connect.JPG?raw=true "Azure Bastion connect")
+![Alt text2](/examples/bastion/images/jumpbox_bastion_ssh.png?raw=true "Azure Bastion Linux SSH")
 
-Click the bastion option and login using your private keys.
+After selecting "Connect" at the bottom of the pane, a new browser tab will open with the jumpbox Linux terminal.
 
-![Alt text2](/examples/bastion/images/bastion_ssh.JPG?raw=true "Azure Bastion ssh")
+## Step 6 - Access the Windows VM via RDP
 
-Should now be on your linux headnode
+The Windows VM can be accessed via RDP exclusively from Azure Portal.
 
-![Alt text3](/examples/bastion/images/bastion_ssh_login.JPG?raw=true "Azure Bastion ssh login")
+Locate the `bastion-winbox` VM on the Azure portal and click on "Connect" menu button. Select the "Bastion" option.
 
+![Alt text3](/examples/bastion/images/winbox_connect.png?raw=true "Windows VM Connect menu button")
 
-Similarly, you can use Azure bastion to login to a Windows VM using RDP
+In the Bastion pane type `hpcadmin` in the "Username" field and select "Password from Azure Key Vault" as authentication type. In the three new drop down menus select the Key Vault deployed by AzureHPC and finally `WinVM-hpcadmin` as secret name.
 
-![Alt text4](/examples/bastion/images/bastion_rdp_windows.JPG?raw=true "Azure Bastion rdp windows")
+![Alt text4](/examples/bastion/images/winbox_bastion_rdp.png?raw=true "Azure Bastion Windows RDP")
 
+After selecting "Connect" at the bottom of the pane, the Windows desktop will be accessible in a new browser tab.

examples/bastion/cloud-init-ubuntu.txt → examples/bastion/templates/cloud-init-ubuntu.txt

@@ -6,4 +6,4 @@ packages:
 runcmd:
   - [ su, "-c", "git clone https://github.com/Azure/azurehpc.git", "-", hpcadmin ]
   - [ sh, "-c", "~hpcadmin/azurehpc/scripts/install-azcopy.sh" ]
-  - [ sh, "-c", "~hpcadmin/azurehpc/examples/bastion/install-azcli-ubuntu.sh" ]
+  - [ sh, "-c", "~hpcadmin/azurehpc/examples/bastion/scripts/install-azcli-ubuntu.sh" ]