Skip to content

Fix/kubernetes devcontainer template to require fewer updates on initial deployment #386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Fix/kubernetes devcontainer template to require fewer updates on initial deployment #386

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0d9ddaa
Added usage of ENVBUILDER_GIT_SSH_PRIVATE_KEY_BASE64 to support ssh …
Aug 25, 2025
4766367
Commented out CPU & RAM Usage metadata due to issues when used on AWS…
Aug 25, 2025
13033de
Changed default namespace to coder, to match install doco and fixed c…
Aug 25, 2025
1971941
Merge branch 'main' into fix/kubernetes-devcontainer-issue
greg-the-coder Aug 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 23 additions & 19 deletions registry/coder/templates/kubernetes-devcontainer/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ variable "use_kubeconfig" {

variable "namespace" {
type = string
default = "default"
default = "coder"
description = "The Kubernetes namespace to create workspaces in (must exist prior to creating workspaces). If the Coder host is itself running as a Pod on the same Kubernetes cluster as you are deploying workspaces to, set this to the same namespace."
}

Expand All @@ -62,7 +62,7 @@ data "coder_parameter" "cpu" {
display_name = "CPU"
description = "CPU limit (cores)."
default = "2"
icon = "/emojis/1f5a5.png"
icon = "/icon/memory.svg"
mutable = true
validation {
min = 1
Expand Down Expand Up @@ -161,6 +161,8 @@ locals {
# ENVBUILDER_GIT_URL and ENVBUILDER_CACHE_REPO will be overridden by the provider
# if the cache repo is enabled.
"ENVBUILDER_GIT_URL" : var.cache_repo == "" ? local.repo_url : "",
# Used for when SSH is an available authentication mechanism for git providers
"ENVBUILDER_GIT_SSH_PRIVATE_KEY_BASE64" : base64encode(try(data.coder_workspace_owner.me.ssh_private_key, "")),
# Use the docker gateway if the access URL is 127.0.0.1
"ENVBUILDER_INIT_SCRIPT" : replace(coder_agent.main.init_script, "/localhost|127\\.0\\.0\\.1/", "host.docker.internal"),
"ENVBUILDER_FALLBACK_IMAGE" : data.coder_parameter.fallback_image.value,
Expand Down Expand Up @@ -263,8 +265,9 @@ resource "kubernetes_deployment" "main" {
name = "dev"
image = var.cache_repo == "" ? local.devcontainer_builder_image : envbuilder_cached_image.cached.0.image
image_pull_policy = "Always"
security_context {}

security_context {
privileged = true
}
# Set the environment using cached_image.cached.0.env if the cache repo is enabled.
# Otherwise, use the local.envbuilder_env.
# You could alternatively write the environment variables to a ConfigMap or Secret
Expand Down Expand Up @@ -352,21 +355,22 @@ resource "coder_agent" "main" {
# if you don't want to display any information.
# For basic resources, you can use the `coder stat` command.
# If you need more control, you can write your own script.
metadata {
display_name = "CPU Usage"
key = "0_cpu_usage"
script = "coder stat cpu"
interval = 10
timeout = 1
}

metadata {
display_name = "RAM Usage"
key = "1_ram_usage"
script = "coder stat mem"
interval = 10
timeout = 1
}
# Note: May not work on AWS Linux Nodes
# metadata {
# display_name = "CPU Usage"
# key = "0_cpu_usage"
# script = "coder stat cpu"
# interval = 10
# timeout = 1
# }
# Note: May not work on AWS Linux Nodes
# metadata {
# display_name = "RAM Usage"
# key = "1_ram_usage"
# script = "coder stat mem"
# interval = 10
# timeout = 1
# }
Comment on lines +358 to +373
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Note: May not work on AWS Linux Nodes
# metadata {
# display_name = "CPU Usage"
# key = "0_cpu_usage"
# script = "coder stat cpu"
# interval = 10
# timeout = 1
# }
# Note: May not work on AWS Linux Nodes
# metadata {
# display_name = "RAM Usage"
# key = "1_ram_usage"
# script = "coder stat mem"
# interval = 10
# timeout = 1
# }
# Note: May not work on AWS Linux Nodes
metadata {
display_name = "CPU Usage"
key = "0_cpu_usage"
script = "coder stat cpu"
interval = 10
timeout = 1
}
# Note: May not work on AWS Linux Nodes
metadata {
display_name = "RAM Usage"
key = "1_ram_usage"
script = "coder stat mem"
interval = 10
timeout = 1
}

@greg-the-coder Would it maybe be worth just adding the Note and leaving the stats. Since not all people would be using kubernetes with aws?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

70%+ of our enterprise, paying customers run Coder on AWS. Also, I ran into the same issue when working with the SUSE/Rancher folks earlier in the year when I was helping them prep a Devcontainer demo for SUSECON. I would prefer to leave that commented out, and then someone could uncomment it if they wanted it, so it at least works out of the box. What I'm seeing in the field on a couple of the customer POVs I've shadowed is the reverse, where we have to walk customers through "tweaking" the templates before they work, which isn't ideal.

Copy link
Member

@matifali matifali Aug 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we instead fix the issue with coder stat mem|cpu and ensure it works in AWS. Can you open an issue in coder/clistat repo? Please include any error messages or screenshots you get.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This template is a general purpose for any kubernetes setup so I would advise against commenting out this section.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really need something that is going to work in the majority of cases, without forcing customers to modify. I've found two cases (SUSE/RKE + AWS) where it doesn't work. My guess is that this only works if you are using Ubuntu for your Node group OS.

Copy link
Member

@matifali matifali Aug 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ia a genuine concern and I agree. Can you please file an issue for this. Meanwhile we can comment this out.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing, I appreciate you and your teams help and support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please link the created issue in comment and Good to merge then.


metadata {
display_name = "Workspaces Disk"
Expand Down