Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster.

This operator is not an operator for a particular software product as prometheus-operator or kafka-operator. Shell-operator provides an integration layer between Kubernetes cluster events and shell scripts by treating scripts as hooks triggered by events. Think of it as an operator-sdk but for scripts.

Shell-operator is used as a base for more advanced Addon-operator that supports helm charts and value storages.

Shell-operator provides:

• Ease the management of a Kubernetes cluster: use the tools that Ops are familiar with. It can be bash, python, kubectl, etc.
• Kubernetes object events: hook can be triggered by add, update or delete events. Learn more about hooks.
• Object selector and properties filter: Shell-operator can monitor only particular objects and detect changes in their properties.
• Simple configuration: hook binding definition is a JSON structure on stdout.

Quickstart

You need to have a Kubernetes cluster, and the kubectl must be configured to communicate with your cluster.

Steps to setup Shell-operator in your cluster are:

• build an image with your hooks (scripts)
• create necessary RBAC objects (for kubernetes bindings)
• run Pod or Deployment with a built image

Build an image with your hooks

A hook is a script that, when executed with --config option, returns configuration in JSON. Learn more about hooks.

Let's create a small operator that will watch for all Pods in all Namespaces and simply log a name of a new Pod.

"kubernetes" binding is used to tell Shell-operator about objects that we want to watch. Create the pods-hook.sh file with the following content:

#!/usr/bin/env bash

if [[ $1 == "--config" ]] ; then
  cat <<EOF
  {
    "configVersion":"v1",
    "kubernetes": [
      {
        "apiVersion": "v1",
        "kind": "Pod",
        "watchEvent": ["Added"]
      }
    ]
  }
EOF
else
  podName=$(jq -r .[0].object.metadata.name $BINDING_CONTEXT_PATH)
  echo "Pod '${podName}' added"
fi

Make the pods-hook.sh executable:

chmod +x pods-hook.sh

You can use a prebuilt image flant/shell-operator:latest with bash, kubectl, jq and shell-operator binaries to build you own image. You just need to ADD your hook into /hooks directory in the Dockerfile.

Create the following Dockerfile in the directory where you created the pods-hook.sh file:

FROM flant/shell-operator:latest
ADD pods-hook.sh /hooks

Build image (change image tag according your Docker registry):

docker build -t "registry.mycompany.com/shell-operator:monitor-pods" .

Push image to the Docker registry accessible by Kubernetes cluster:

docker push registry.mycompany.com/shell-operator:monitor-pods

Install shell-operator in a cluster

We need to watch for Pods in all Namespaces. That means that we need specific RBAC definitions for shell-operator:

kubectl create namespace example-monitor-pods &&
kubectl create serviceaccount monitor-pods-acc \
  --namespace example-monitor-pods &&
kubectl create clusterrole monitor-pods --verb=get,watch,list --resource=pods &&
kubectl create clusterrolebinding monitor-pods \
  --clusterrole=monitor-pods \
  --serviceaccount=example-monitor-pods:monitor-pods-acc

Shell-operator can be deployed as a Pod. Put this manifest into the shell-operator-pod.yaml file:

apiVersion: v1
kind: Pod
metadata:
  name: shell-operator
spec:
  containers:
  - name: shell-operator
    image: registry.mycompany.com/shell-operator:monitor-pods
    imagePullPolicy: Always
  serviceAccountName: monitor-pods-acc

Start shell-operator by applying a shell-operator-pod.yaml file:

kubectl -n example-monitor-pods apply -f shell-operator-pod.yaml

For instance, deploy kubernetes-dashboard to trigger kubernetes hook:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml

Run kubectl -n example-monitor-pods logs po/shell-operator and see that the hook will print dashboard pod names:

...
INFO     : QUEUE add TASK_HOOK_RUN@KUBERNETES pods-hook.sh
INFO     : TASK_RUN HookRun@KUBERNETES pods-hook.sh
INFO     : Running hook 'pods-hook.sh' binding 'kubernetes' ...
Pod 'kubernetes-dashboard-769df5545f-99xsb' added
...

To delete created objects execute:

kubectl delete ns example-monitor-pods &&
kubectl delete clusterrole monitor-pods &&
kubectl delete clusterrolebinding monitor-pods

This example is also available in /examples: monitor-pods.

Hook binding types

onStartup

This binding has only one parameter: order of execution. Hooks are loaded at start and then hooks with onStartup binding are executed in order defined by parameter. Read more about onStartup bindings here.

Example hook --config:

{
  "configVersion": "v1",
  "onStartup":10
}

schedule

This binding is for periodical running of hooks. Schedule can be defined with granularity of seconds. Read more about schedule bindings here.

Example hook --config with 2 schedules:

{
  "configVersion": "v1",
  "schedule": [
    {
      "name": "every 10 min",
      "crontab": "0 */10 * * * *",
      "allowFailure": true
    },
    {
      "name": "Every Monday at 8:05",
      "crontab": "5 8 * * 1"
    }
  ]
}

kubernetes

This binding defines a subset of Kubernetes objects that Shell-operator will monitor and a jq expression to filter their properties. Read more about onKubernetesEvent bindings here.

Example of hook --config:

{
  "configVersion": "v1",
  "kubernetes": [
    {
      "name":"Execute on changes of namespace labels",
      "kind": "namespace",
      "watchEvent":["Modified"],
      "jqFilter":".metadata.labels"
    }
  ]
}

Note: it is possible to watch custom resources, just use proper values for apiVersion and kind fields.

Prometheus target

Shell-operator provides /metrics endpoint. More on this in METRICS document.

Examples

More examples can be found in examples directory.

License

Apache License 2.0, see LICENSE.