Skip to content

Commit

Permalink
feat(superchain): arm64 support (aws#2949)
Browse files Browse the repository at this point in the history 
Introducing ARM64 support for the `jsii/superchain` docker image using
the multi-arch capabilities of `docker buildx`. This prompted a couple
of changes in the image, which should be fine for the majority of
use-cases, but is a breaking change in certain edge cases... So this change
also changes the image tagging model, so that users are able to reliably
depend on a particular linux distribution ancestry (`debian:buster-slim`
being the only offered option at this moment).

The new image is based off `debian:10-slim`, as unfortunately, binary
distributions of some of the image contents are not available for ARM64
platform on the RHEL/CentOS-based distributions (which `amazonlinux:2`
is). This means packages now get installed from `apt` instead of `yum`.

Image testing (by building `jsii` within the image) is now part of the
image build process itself, instead of being separated, as this allows
testing on all supported architectures (which is otherwise somewhat more
difficult to achieve).

BREAKING CHANGE: the `jsii/superchain` image tags `:latest`, `:nightly`,
`:node10` and `:node14` are no longer maintained. Users should migrate to
the new debian-based tags starting with `:1-buster-slim`.

Fixes aws#2930
  • Loading branch information
@RomainMuller
RomainMuller authored Aug 16, 2021
1 parent e729f5d commit ceb66a6
Show file tree
Hide file tree
Showing 9 changed files with 800 additions and 134 deletions.
158 changes: 132 additions & 26 deletions .github/workflows/docker-images.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ on:
push:
branches: [main, release]

env:
DOCKER_BUILDKIT: 1

jobs:
superchain:
name: jsii/superchain
Expand All @@ -17,8 +20,8 @@ jobs:
matrix:
node: ['10', '14']
env:
# Node version whose images will be aliased to 'nightly' and 'latest'
DEFAULT_NODE_MAJOR_VERSION: 10
# Node version whose images will be aliased without the -nodeXX segment
DEFAULT_NODE_MAJOR_VERSION: 12
steps:
- name: Check out
uses: actions/[email protected]
Expand Down Expand Up @@ -58,48 +61,151 @@ jobs:
# Ensure we run with bash, because that's the syntax we're using here...
shell: bash

- name: Set up QEMU
uses: docker/setup-qemu-action@v1
with:
platforms: arm64

- name: Set up docker buildx
id: buildx
if: steps.should-run.outputs.result == 'true'
uses: docker/setup-buildx-action@v1

# We only restore GH cache if we are not going to publish the result (i.e: PR validation)
- name: Set up layer cache
if: steps.should-run.outputs.result == 'true' && github.event_name != 'push'
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ hashFiles('superchain/*') }}-${{ github.sha }}
restore-keys: |-
${{ runner.os }}-buildx-${{ hashFiles('superchain/*') }}-
${{ runner.os }}-buildx-
- name: Determine build time
id: build-time
if: steps.should-run.outputs.result == 'true'
run: |-
echo "::set-output name=value::$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
- name: Build Image
if: steps.should-run.outputs.result == 'true'
run: |-
docker build \
--pull \
--build-arg BUILD_TIMESTAMP="$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
--tag "jsii/superchain:node${{ matrix.node }}-nightly" \
./superchain
- name: Test Image
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64,linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache-out \
--pull \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
-f superchain/Dockerfile \
.
# Replace the cache so it does not grow forever
- name: Update layer cache
if: always() && steps.should-run.outputs.result == 'true'
run: |-
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-out /tmp/.buildx-cache
# Testing sequentially, because in parallel it's too slow due to IO contention
- name: Test Image (AMD64)
if: steps.should-run.outputs.result == 'true'
run: |-
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
-f superchain/Dockerfile \
.
- name: Test Image (ARM64)
if: steps.should-run.outputs.result == 'true'
run: |-
docker run \
--rm \
--tty \
--network=host \
-v${{ github.workspace }}:${{ github.workspace }} \
-w${{ github.workspace }} \
"jsii/superchain:node${{ matrix.node }}-nightly" \
bash -c "yarn install --frozen-lockfile && yarn build && yarn test"
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
-f superchain/Dockerfile \
.
# Only when puhsing to main/release from now on
- name: Publish (nightly)
if: steps.should-run.outputs.result == 'true' && github.event_name == 'push' && github.ref != 'refs/heads/release'
run: |-
docker push jsii/superchain:node${{ matrix.node }}-nightly
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64,linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--push \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
--tag "jsii/superchain:1-buster-slim-node${{ matrix.node }}-nightly"\
-f superchain/Dockerfile \
.
# If the current version is the default version, also tag this with the unqualified ':nightly' label
if [[ "${{ matrix.node }}" == "$DEFAULT_NODE_MAJOR_VERSION" ]]; then
docker tag jsii/superchain:node${{ matrix.node }}-nightly jsii/superchain:nightly
docker push jsii/superchain:nightly
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64,linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--push \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
--tag "jsii/superchain:1-buster-slim-nightly" \
-f superchain/Dockerfile \
.
fi
- name: Publish (latest)
if: steps.should-run.outputs.result == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/release'
run: |-
# Alias 'nodeX-nightly's to 'nodeX', then push
docker tag jsii/superchain:node${{ matrix.node }}-nightly jsii/superchain:node${{ matrix.node }}
docker push jsii/superchain:node${{ matrix.node }}
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64,linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--push \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
--tag "jsii/superchain:1-buster-slim-node${{ matrix.node }}" \
-f superchain/Dockerfile \
.
# If the current version is the default version, also tag this with the unqualified ':latest' label
# If the current version is the default version, also tag this with the unqualified ':1-*' label
if [[ "${{ matrix.node }}" == "$DEFAULT_NODE_MAJOR_VERSION" ]]; then
docker tag jsii/superchain:node${{ matrix.node }} jsii/superchain:latest
docker push jsii/superchain:latest
docker buildx build \
--builder ${{ steps.buildx.outputs.name }} \
--platform linux/amd64,linux/arm64 \
--target superchain \
--cache-from type=local,src=/tmp/.buildx-cache \
--cache-to type=local,dest=/tmp/.buildx-cache \
--push \
--build-arg BUILD_TIMESTAMP="${{ steps.build-time.outputs.value }}" \
--build-arg COMMIT_ID='${{ github.sha }}' \
--build-arg NODE_MAJOR_VERSION=${{ matrix.node }} \
--tag "jsii/superchain:1-buster-slim" \
-f superchain/Dockerfile \
.
fi
2 changes: 2 additions & 0 deletions superchain/.dockerignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/.dockerignore
/build-local.sh
Loading

0 comments on commit ceb66a6

Please sign in to comment.