Skip to content

Commit

Permalink
build: make docker builds faster with cargo-chef (paradigmxyz#1432)
Browse files Browse the repository at this point in the history
  • Loading branch information
onbjerg authored Feb 17, 2023
1 parent 7c9b212 commit e24a1dd
Show file tree
Hide file tree
Showing 5 changed files with 50 additions and 75 deletions.
33 changes: 22 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,23 +1,34 @@
# Use rust image as the builder base
FROM rust:1.66.0-bullseye AS builder
FROM lukemathwalker/cargo-chef:latest-rust-1 AS chef
WORKDIR app

# Builds a cargo-chef plan
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json

# Set the build profile to be release
ARG BUILD_PROFILE=release
ENV BUILD_PROFILE $BUILD_PROFILE

# Update and install dependencies
# Install system dependencies
RUN apt-get update && apt-get -y upgrade && apt-get install -y libclang-dev pkg-config

# Copy base folder into docker context
COPY . reth
# Builds dependencies
RUN cargo chef cook --profile $BUILD_PROFILE --recipe-path recipe.json

# Build reth
RUN cd reth && cargo build --all --locked --profile $BUILD_PROFILE
# Build application
COPY . .
RUN cargo build --profile $BUILD_PROFILE --locked --bin reth

# Use Ubuntu as the release image
FROM ubuntu
FROM ubuntu AS runtime
WORKDIR app

# Copy the built reth binary from the previous stage
COPY --from=builder /reth/target/release/reth /usr/local/bin/reth
# Copy reth over from the build stage
COPY --from=builder /app/target/release/reth /usr/local/bin

CMD ["/usr/local/bin/reth"]
EXPOSE 30303 30303/udp 9000 8545 8546
ENTRYPOINT ["/usr/local/bin/reth", "node"]
61 changes: 15 additions & 46 deletions book/getting_started/installation.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,30 @@
## Installation

To use Reth, you must first install Geth. You can find instructions for installing Geth at the following link: [https://geth.ethereum.org/docs/install-and-build/installing-geth](https://geth.ethereum.org/docs/getting-started/installing-geth).
reth is currently unstable and does not have officially cut releases. To install reth, you must either install it from source or using Docker.

### Ubuntu
Building the source
### From Source

* install rustup
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
To build from source, first install [Rust](https://rustup.rs). You also need some basic build tools for our C libraries:

* install Requirements
- libclang-dev
- pkg-config
- For Ubuntu: `apt-get install libclang-dev pkg-config`
- For Arch: `pacman -S base-devel`
- For macOS: `brew install llvm pkg-config`

```bash
apt install libclang-dev pkg-config
```
* Build reth
```bash
Then clone the repository and build the binary:

```console
git clone https://github.com/paradigmxyz/reth
cd reth
cargo build --all
./target/debug/reth
cargo install --release --locked --path . --bin reth
```

### MacOS
The binary will now be in a platform specific folder, and should be accessible as `reth` via the command line.

To install and build reth on macOS using Homebrew, you can use the following steps
### Using Docker

* Install rustup by running the following command in a terminal:
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
Clone the repository and build the image:

* Install the necessary requirements by running the following command:
```bash
brew install llvm pkg-config
```

* Clone the reth repository and navigate to the directory:
```bash
```console
git clone https://github.com/paradigmxyz/reth
cd reth
```

* Build reth using cargo:
```bash
cargo build --all
```

* Run reth:
```bash
./target/debug/reth
```

* Alternatively, you can use the following one-liner to install and build reth:
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && brew install llvm pkg-config && git clone https://github.com/paradigmxyz/reth && cd reth && cargo build --all && ./target/debug/reth
docker build -t paradigmxyz/reth .
```
4 changes: 2 additions & 2 deletions book/getting_started/introduction.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Intruduction
# Introduction

Reth is a user-friendly, modular, fast, and efficient Ethereum full node implementation that is compatible with the [Engine API](https://github.com/ethereum/execution-apis/tree/main/src/engine). It allows users to connect to the Ethereum network and interact with the blockchain, including sending and receiving transactions and accessing smart contracts. Reth is built and maintained by [Paradigm](https://www.paradigm.xyz/) and is licensed under the Apache and MIT licenses.

## Status

The project is not ready for use. We hope to have full sync implemented sometime in January/February 2023, followed by optimizations. In the meantime, we're working on making sure every crate of the repository is well documented, abstracted and tested.
The project is not ready for use. We hope to have full sync implemented sometime in January/February 2023, followed by optimizations. In the meantime, we're working on making sure every crate of the repository is well documented, abstracted and tested.
22 changes: 10 additions & 12 deletions book/getting_started/running.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,20 @@ This chapter will provide a few runbooks for getting the node up and running.

This goal of this is not to expose all of the options available to you for running Reth (e.g. config file options, CLI flags, etc.) - those will be documented in the other chapters - but rather showcase some examples of different ways in which one could run the node.

## Run on MacOS w/ Prometheus + Grafana for Metrics
## Run on w/ Prometheus + Grafana for Metrics

This will provide a brief overview for how to run Reth locally on a Mac.
This will provide a brief overview for how to run Reth locally.

First, ensure that you have Reth installed by following the [instructions to install on Mac][mac-installation].
First, ensure that you have Reth installed by following the [installation instructions][installation].

### Basic operation

The most basic way to run it is by using the following command:

```bash
cargo run --release -- node
```console
reth node
```

This will build Reth using cargo's [release profile][release-profile], and run it with an error log level by default.

You will likely see a large number of error logs to the tune of:

```bash
Expand All @@ -35,7 +33,7 @@ When playing around with sync, you may want to set a threshold block that you'd
To do so, add the `--debug.tip` flag with the block hash that you'd like to sync to:

```bash
cargo run --release -- node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
```

This will sync to block 1M.
Expand All @@ -47,7 +45,7 @@ That's great and all, but wouldn't it be nice to get a deeper view into what's g
Let's try again, now with the following command:

```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace cargo run --release -- node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
RUST_LOG=info,sync::stages=trace,downloaders=trace reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
```

This will dump info-level logs throughout the node's operation, as well as trace-level logs for the pipeline stages and the downloaders (which fetch data over the P2P network). Check out the [docs][docs] for more info on these!
Expand All @@ -67,7 +65,7 @@ These warnings are also nothing to worry about, all of this is part of the norma
You may want to keep these logs around outside of your terminal. To accomplish this, let's run:

```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace cargo r --release -- node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --log.directory ./
RUST_LOG=info,sync::stages=trace,downloaders=trace reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --log.directory ./
```

Here, adding `--log.directory` specifies a location to which the logs will be saved (in a file named `reth.log`) so that you can view them with a tool like `more`, `less`, or `tail`.
Expand All @@ -79,7 +77,7 @@ Now, trying to get a sense of sync progress by scanning through the logs is quit
Reth exposes a number of metrics, which are listed [here][metrics]. We can serve them from an HTTP endpoint by adding the `--metrics` flag:

```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace nohup cargo r --release -- node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --metrics '127.0.0.1:9000' > reth-out.txt &
RUST_LOG=info,sync::stages=trace,downloaders=trace nohup reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --metrics '127.0.0.1:9000' > reth-out.txt &
```

Now, as the node is running, you can `curl` the endpoint you provided to the `--metrics` flag to get a text dump of the metrics at that time:
Expand Down Expand Up @@ -141,7 +139,7 @@ In this runbook, we took you through starting the node, exposing different log l

This will all be very useful to you, whether you're simply running a home node and want to keep an eye on its performance, or if you're a contributor and want to see the effect that your (or others') changes have on Reth's operations.

[mac-installation]: ./installation.md#macos
[installation]: ./installation.md
[release-profile]: https://doc.rust-lang.org/cargo/reference/profiles.html#release
[docs]: https://github.com/paradigmxyz/reth/tree/main/docs
[metrics]: https://github.com/paradigmxyz/reth/blob/main/docs/design/metrics.md#current-metrics
5 changes: 1 addition & 4 deletions book/getting_started/system_requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,13 @@
- We recommend a minimum of 8GB RAM
- We recommend using an SSD or NVMe, as an HDD may cause the node to fall behind the chain tip. Note that SSD performance can deteriorate when close to capacity.


### Archive node
- TODO: We will update the disk usage estimate once it is ready.

### Full node
- TODO: We will update the disk usage estimate once it is ready.



## Software
## Software (for building)
- rustc 1.66.0 (69f9c33d7 2022-12-12)
- libclang-dev
- pkg-config

0 comments on commit e24a1dd

Please sign in to comment.