Feature request - adding additional fake nodes

[katapultcloud]

Hey guys,
I have a question regarding adding additional fake nodes into vcluster clusters due to testing taints / labels scheduling etc. My cluster is small 1 cp + 1-2 workers, but I'd like to simulate my multi tenants to have more nodes. I was wondering if vcluster comes with that feature as I couldn't find it anywhere.

on host

$ k get node
NAME          STATUS   ROLES           AGE   VERSION
cp            Ready    control-plane   51m   v1.27.1
worker-0      Ready    <none>          48m   v1.27.1

on vcluster right now

$ k --kubeconfig=kubeconfig.yaml get node
NAME       STATUS   ROLES    AGE   VERSION
worker-0   Ready    <none>   32m   v1.27.2+k3s1

ideal situation

$ k --kubeconfig=kubeconfig.yaml get node
NAME       STATUS   ROLES    AGE   VERSION
worker-0   Ready    <none>   32m   v1.27.2+k3s1
worker-1   Ready    <none>   32m   v1.27.2+k3s1
worker-2   Ready    <none>   32m   v1.27.2+k3s1
worker-3   Ready    <none>   32m   v1.27.2+k3s1
worker-4   Ready    <none>   32m   v1.27.2+k3s1

Thanks
Stefan

[matskiv]

vcluster does not have such a feature at the moment. The "fake nodes" mentioned in the docs are IMHO an unfortunate term, and I've seen people confusing it with what you are requesting.
This came up a few times, so let's keep this feature request up.

[katapultcloud]

Is there any way we can simulate adding fake nodes right now? I'm trying to figure out a way and experimenting with adding additional tls-san with pod name and running k3s agent as a pod:

apiVersion: v1
kind: Pod
metadata:
  name: node-0
spec:
  containers:
    - env:
        - name: K3S_TOKEN
          value: albert
        - name: K3S_URL
          value: https://admin
      image: rancher/k3s:latest
      args:
        - "agent"
      name: node-0
      resources: {}
      securityContext:
        privileged: true
      volumeMounts:
        - mountPath: /run
          name: tmpfs0
        - mountPath: /var/run
          name: tmpfs1
  restartPolicy: Always
  volumes:
    - emptyDir:
        medium: Memory
      name: tmpfs0
    - emptyDir:
        medium: Memory
      name: tmpfs1

but I'm getting forbidden from k3s agent logs

time="2023-06-21T18:50:28Z" level=error msg="failed to get CA certs: https://127.0.0.1:6444/cacerts: 403 Forbidden"
time="2023-06-21T18:50:30Z" level=error msg="failed to get CA certs: https://127.0.0.1:6444/cacerts: 403 Forbidden"
time="2023-06-21T18:50:32Z" level=error msg="failed to get CA certs: https://127.0.0.1:6444/cacerts: 403 Forbidden"
time="2023-06-21T18:50:34Z" level=error msg="failed to get CA certs: https://127.0.0.1:6444/cacerts: 403 Forbidden"

[matskiv]

I don't think it is possible to achieve this now without some code changes in vcluster.

[ThomasK33]

Hey @katapultcloud, have you considered using KWOK or the Kubernetes Scheduler Simulator for emulating and testing scheduling of workloads?

Those tools might be more adequate than vCluster.

[katapultcloud]

@ThomasK33 sorry for late response I must've missed the notification. Actually these look great, but I would still need the functionality of vCluster with additional nodes. In the mean time I have migrated to a completely different solution Kubevirt with Kubeadm clusters. For my used case this was the best option so far. I'll keep this one open as it would be nice to have this functionality with vCluster.