Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

toolchain in recipe.json is ignored in cargo chef cook #271

Open
kobkaz opened this issue Jul 1, 2024 · 1 comment · May be fixed by #281
Open

toolchain in recipe.json is ignored in cargo chef cook #271

kobkaz opened this issue Jul 1, 2024 · 1 comment · May be fixed by #281

Comments

@kobkaz
Copy link

kobkaz commented Jul 1, 2024

cargo chef cook generates rust-toolchain.toml file, but it's not used in the subsequent build of dependencies.
rust-toolchain.toml takes effect when cargo is invoked next time. Therefore, executing cargo chef cook twice results in building with the wrong toolchain for the first time and the correct building for the second time.

The following Dockerfile reproduces the problem.

FROM lukemathwalker/cargo-chef:latest-rust-1 AS chef
WORKDIR /app

FROM chef AS planner
RUN echo '[workspace]\nresolver = "2"\nmembers = ["hello"]' > Cargo.toml
RUN cargo new hello
RUN echo '[toolchain]\nchannel = "1.75.0"' > rust-toolchain.toml
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json # does not install 1.75.0
RUN cargo chef cook --release --recipe-path recipe.json # does install 1.75.0
@LtdSauce LtdSauce mentioned this issue Oct 5, 2024
Closed
1 task
@LtdSauce
Copy link

LtdSauce commented Oct 5, 2024 •
edited
Loading

I applied a work-around for this by manually copying the toolchain file before invoking cargo chef cook.
Assuming the toolchain file is located in the build-root, just add the following line before the invocation of cook:

COPY ./rust-toolchain.toml rust-toolchain.toml

If there are any hidden implications to this i do not know... but at least it gets used during the builds in the cook step :)

PS: This might actually be connected to #231

This was referenced Oct 5, 2024
Closed
Open
LtdSauce added a commit to LtdSauce/git-cliff that referenced this issue Oct 11, 2024
@LtdSauce
chore(docker): ignore rust toolchain in docker builds
7b1bf94 
This commit adds the known names of the rust-toolchain files to the
.dockerignore file. This has three reasons why it makes sense:

- The initial docker layer already has a set up rust toolchain that is
  sufficient to build the project. Thus, by providing a toolchain file,
  the toolchain would be installed again during docker build.
- Currently cargo-chef only copies the toolchain files during cooking
  but it gets not used during the building of the dependencies in the
  cook call, see
  LukeMathWalker/cargo-chef#271.
  With this in mind, currently the dependencies were actually build
  twice. Once with the installed toolchain from the image itself, and
  then in the actual cargo build call with the toolchain speciefied in
  the toolchain file. Building them twice resulted in timeouts when
  building the arm64 images as they are emulated using qemu, which is
  itself already slower than building natively.

Now one could argue, that as soon as the mentioned issue is solved using
the toolchain again would be fine. But then it would be still needed to
assemble the Dockerfile in a way that the toolchain is not build twice.
Because the current structure of the Dockerfile builds the toolchain
once in the cargo-chef prepare step and once during the cargo build step
(and would later build it during the cargo-chef cook instead of cargo
build).

With all this in mind using no toolchain file but instead just using
the sufficient rust installation from the base image makes sense.
LtdSauce added a commit to LtdSauce/git-cliff that referenced this issue Oct 14, 2024
@LtdSauce
chore(docker): ignore rust toolchain in docker builds
73f75d5 
This commit adds the known names of the rust-toolchain files to the
.dockerignore file. This has two reasons why it makes sense:

- The initial docker layer already has a set up rust toolchain that is
  sufficient to build the project. Thus, by providing a toolchain file,
  the toolchain would be installed again during docker build.
- Currently cargo-chef only copies the toolchain files during cooking
  but it gets not used during the building of the dependencies in the
  cook call, see
  LukeMathWalker/cargo-chef#271.
  With this in mind, currently the dependencies were actually build
  twice. Once with the installed toolchain from the image itself, and
  then in the actual cargo build call with the toolchain speciefied in
  the toolchain file. Building them twice resulted in timeouts when
  building the arm64 images as they are emulated using qemu, which is
  itself already slower than building natively.

Now one could argue, that as soon as the mentioned issue is solved using
the toolchain again would be fine. But then it would be still needed to
assemble the Dockerfile in a way that the toolchain is not build twice.
Because the current structure of the Dockerfile builds the toolchain
once in the cargo-chef prepare step and once during the cargo build step
(and would later build it during the cargo-chef cook instead of cargo
build).

With all this in mind using no toolchain file but instead just using
the sufficient rust installation from the base image makes sense.
orhun pushed a commit to orhun/git-cliff that referenced this issue Oct 17, 2024
@LtdSauce
feat(docker): build arm64 images again (#879) (#919)
84771f6 
* chore(docker): ignore rust toolchain in docker builds

This commit adds the known names of the rust-toolchain files to the
.dockerignore file. This has two reasons why it makes sense:

- The initial docker layer already has a set up rust toolchain that is
  sufficient to build the project. Thus, by providing a toolchain file,
  the toolchain would be installed again during docker build.
- Currently cargo-chef only copies the toolchain files during cooking
  but it gets not used during the building of the dependencies in the
  cook call, see
  LukeMathWalker/cargo-chef#271.
  With this in mind, currently the dependencies were actually build
  twice. Once with the installed toolchain from the image itself, and
  then in the actual cargo build call with the toolchain speciefied in
  the toolchain file. Building them twice resulted in timeouts when
  building the arm64 images as they are emulated using qemu, which is
  itself already slower than building natively.

Now one could argue, that as soon as the mentioned issue is solved using
the toolchain again would be fine. But then it would be still needed to
assemble the Dockerfile in a way that the toolchain is not build twice.
Because the current structure of the Dockerfile builds the toolchain
once in the cargo-chef prepare step and once during the cargo build step
(and would later build it during the cargo-chef cook instead of cargo
build).

With all this in mind using no toolchain file but instead just using
the sufficient rust installation from the base image makes sense.

* Revert "chore(docker): disable building arm64 docker images temporarily (#879)"

This reverts commit cde2a8e.
Commit 73f75d5 made it possible to
build the arm64 image again without running into timeouts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: use toolchain file while building dependencies LtdSauce/cargo-chef
2 participants
@kobkaz @LtdSauce