Skip to content

add docs on glue k8s executor #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 4, 2025
Merged

add docs on glue k8s executor #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/content/docs/aws/capabilities/config/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,14 @@ Also see [OpenSearch configuration variables](#opensearch) which are used to man
| - | - | - |
| `PROVIDER_OVERRIDE_EVENTS` | `legacy`\|`v2` (default) | The [new EventBridge provider](https://discuss.localstack.cloud/t/introducing-eventbridge-v2-in-localstack/946) is active by default since LocalStack 4.0. |

### Glue

| Variable | Example Values | Description |
| - | - | - |
| `GLUE_JOB_EXECUTOR_PROVIDER` | `v1` (default) \| `v2` | Feature flag to switch between the old and new Glue job executor providers. |
| `GLUE_JOB_EXECUTOR` | `docker` (default) \| `kubernetes` | Whether to run Glue jobs when LocalStack is deployed on Kubernetes. Jobs are run as pods in the Kubernetes cluster. |
| `DOCKER_GLOBAL_IMAGE_PREFIX` | | Specify custom images for Glue jobs by configuring their custom image repository. |

### IAM

| Variable | Example Values | Description |
Expand Down
3 changes: 2 additions & 1 deletion src/content/docs/aws/enterprise/kubernetes-executor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ You can run the following services on Kubernetes clusters using the LocalStack E

- [DocumentDB](/aws/services/docdb)
- [MWAA](/aws/services/docdb)
- [Glue](/aws/services/glue) (requires setting `GLUE_JOB_EXECUTOR_PROVIDER` to `v2`)
- [RDS](/aws/services/rds) ([MySQL](/aws/services/rds/#mysql-engine) & [MSSQL](/aws/services/rds/#microsoft-sql-server-engine))

To use Kubernetes as the runtime backend, set the `CONTAINER_RUNTIME` configuration variable to `kubernetes`.
Note that there are no service-specific configuration variables for these services.
Note that there are no service-specific configuration variables for these services.
1 change: 1 addition & 0 deletions src/content/docs/aws/services/glue.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Key enhancements include:
- Executing multiple jobs in parallel
- Ensuring correct versioning of Spark, Hadoop, Python, Java, and related libraries
- Improving startup times and offline execution support
- Ability to run Glue jobs on Kubernetes clusters

To use it, set `GLUE_JOB_EXECUTOR=docker` and `GLUE_JOB_EXECUTOR_PROVIDER=v2` in your LocalStack configuration.
The new executor additionally deprecates older versions of Glue (`0.9`, `1.0`, `2.0`).
Expand Down