splunk-k8s

Repo for testing Splunk on Kubernetes.

Contents

• Install
  • Build AKS Cluster with Terraform
  • Connect to AKS Cluster
  • Install Splunk Operator
    • Non-admin Installation
  • Install Splunk Deployment
  • Get Admin Password
• View Web Interface (port-forward pods)
• Start/Stop AKS Cluster
• Uninstall
  • Uninstall Splunk Deployment
  • Uninstall Splunk Operator
  • Destroy AKS Cluster

Install

Build AKS Cluster with Terraform

Use Terraform to build the AKS cluster:

# Azure login and set Terraform env vars
source ~/path_to_azure_login_script.sh

# Init
terraform init

# Apply
terraform apply

# Outputs
terraform output

Connect to AKS Cluster

The terraform output command below will return the az cli command required to get aks credentials:

# output the az cli command required to get aks credentials
terraform output aks_credentials_command

Example:
az aks get-credentials --resource-group <AKS_RESOURCE_GROUP_NAME> --name <AKS_CLUSTER_NAME> --overwrite-existing --admin

Install Splunk Operator

The Splunk Operator for Kubernetes (SOK) makes it easy for Splunk Administrators to deploy and operate Enterprise deployments in a Kubernetes infrastructure. Packaged as a container, it uses the operator pattern to manage Splunk-specific custom resources, following best practices to manage all the underlying Kubernetes objects for you.

Read the Getting Started Documentation for more information.

Non-admin Installation

Install the Splunk Operator as a non-admin user, as the Admin Installation for All Namespaces method has an outstanding issue.

# create namespace
kubectl create namespace sok

# an admin needs to install the CRDs
kubectl apply --namespace sok -f https://github.com/splunk/splunk-operator/releases/download/1.0.1/splunk-operator-crds.yaml

# install splunk operator into namespace
kubectl apply --namespace sok -f https://github.com/splunk/splunk-operator/releases/download/1.0.1/splunk-operator-noadmin.yaml

Install Splunk Deployment

First, create a ConfigMap using a license file called enterprise.lic (provide your own license and place in root of repo):

# create license configmap from enterprise.lic
kubectl create configmap splunk-licenses --namespace sok --from-file=enterprise.lic

Deploy a Splunk Validated Architecture from here: https://github.com/splunk/splunk-operator/tree/develop/deploy/examples/advanced

# [option 1] deploy c1 example
kubectl apply --namespace sok -f examples/validated-arch/c1.yaml

# [option 2] deploy c1 custom example
kubectl apply --namespace sok -f examples/validated-arch/c1-custom.yaml

Get Admin Password

After deploying Splunk, view the admin password by running this:

kubectl get secret splunk-sok-secret --namespace sok -o jsonpath='{.data.password}' | base64 --decode

You can also show all global secret values by running the following code:

kubectl get secret --namespace sok splunk-sok-secret -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'

View Web Interface (port-forward pods)

Use port-forwarding on the pods to view their web interface:

# view pods on http://localhost:8000/
# (NOTE: you cannot use the same port more than once)
kubectl port-forward --namespace sok splunk-cm-example-cluster-master-0 8000
kubectl port-forward --namespace sok splunk-lm-example-license-master-0 8000

Start/Stop AKS Cluster

Use the commands below to stop and start the AKS cluster:

# show current aks power state
az aks show --name <AKS_CLUSTER_NAME> --resource-group <AKS_RESOURCE_GROUP_NAME> --query "powerState"

# stop aks cluster
az aks stop --name <AKS_CLUSTER_NAME> --resource-group <AKS_RESOURCE_GROUP_NAME>

# start aks cluster
az aks start --name <AKS_CLUSTER_NAME> --resource-group <AKS_RESOURCE_GROUP_NAME>

Uninstall

Uninstall Splunk Deployment

# delete c1 example
kubectl delete --namespace sok -f examples/validated-arch/c1.yaml

Uninstall Splunk Operator

# delete splunk operator
kubectl delete --namespace sok -f https://github.com/splunk/splunk-operator/releases/download/1.0.1/splunk-operator-noadmin.yaml

# delete CRDs
kubectl delete --namespace sok -f https://github.com/splunk/splunk-operator/releases/download/1.0.1/splunk-operator-crds.yaml

# [optional] delete namespace
kubectl delete namespace sok

If the namespace or other resources are stuck in a terminating state, check for remaining CRD instances, edit their yaml and delete the finalizers key, eg. remove the following:

finalizers:
  - enterprise.splunk.com/delete-pvc

Destroy AKS Cluster

# destroy aks cluster
terraform destroy