Skip to content

Commit

Permalink
Rebuild buildkite-test images (dagster-io#15710)
Browse files Browse the repository at this point in the history
It's been a while since we built and pushed these and we need the node
20 upgrade.

I've played whack-a-mole to get all of these building and pushed them.
I'll run a test build on buildkite to make sure everything works
(particularly because I changed how we install java).

It might be worth switching this to build automatically at some cadence
so it doesn't atrophy as much.
  • Loading branch information
jmsanders authored Aug 8, 2023
1 parent 1b6adb3 commit 81cc66f
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 15 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,13 @@ RUN apt-get update -yqq \
libxmlsec1-dev \
pkg-config

# Install Java (required by pyspark)
COPY --from=openjdk:8-jre-slim /usr/local/openjdk-8 /usr/local/openjdk-8
ENV JAVA_HOME /usr/local/openjdk-8
RUN update-alternatives --install /usr/bin/java java /usr/local/openjdk-8/bin/java 1

# Install various packages used in running/installing/testing Dagster:
# - git/make (cloning dagster, running checks defined in Makefile)
# - openjdk-11-{jdk,jre}-headless 8 (Java required by pyspark) see: http://bit.ly/2ZIuHRh
# - nodejs/yarn (dagster UI)
# - assorted others-- some may no longer be required, but leaving in place for now
#
Expand All @@ -81,8 +85,6 @@ RUN curl -sL https://deb.nodesource.com/setup_20.x | bash - \
make \
mariadb-client \
nodejs \
openjdk-11-jdk-headless \
openjdk-11-jre-headless \
pandoc \
postgresql \
rabbitmq-server \
Expand All @@ -92,9 +94,7 @@ RUN curl -sL https://deb.nodesource.com/setup_20.x | bash - \
unzip \
wget \
xz-utils \
yarn \
# Confirm Java works
&& java -version
yarn

# Install FOSSA cli for analyzing our open-source dependencies
# https://github.com/fossas/fossa-cli
Expand All @@ -108,13 +108,12 @@ RUN wget https://github.com/fossas/fossa-cli/releases/download/v${FOSSA_VERSION}
# Install Google Cloud SDK
# https://cloud.google.com/sdk/docs/install#deb
# Add Google Cloud GPG key and apt source
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg > /usr/share/keyrings/cloud.google.gpg \
&& echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | \
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - \
&& echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | \
tee -a /etc/apt/sources.list.d/google-cloud-sdk.list > /dev/null \
# Install google cloud
&& apt-get -yqq update \
&& apt-get -yqq install \
google-cloud-sdk
google-cloud-cli

# Install Docker -- note that we have some older code using `docker-compose` instead of `docker
# compose`, so we have to install both the `docker-compose` executable and the docker compose CLI
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
'3.10': 2023-03-13T170520
'3.11': 2023-03-13T170520
'3.7': 2023-03-13T170520
'3.8': 2023-03-13T170520
'3.9': 2023-03-13T170520
'3.10': 2023-08-07T143538
'3.11': 2023-08-07T143538
'3.8': 2023-08-07T143538
'3.9': 2023-08-07T143538

0 comments on commit 81cc66f

Please sign in to comment.