Skip to content

Latest commit

 

History

History

node

README.md

Substrate Node Template

A new FRAME-based Substrate node, ready for hacking 🚀

Local Development

Follow these steps to prepare a local Substrate development environment 🛠️

Simple Setup

Install all the required dependencies with a single command (be patient, this can take up to 30 minutes).

curl https://getsubstrate.io -sSf | bash -s -- --fast

Dependencies

If the command above doesn't work, the dependencies can also be installed manually.

sudo apt install -y cmake pkg-config libssl-dev git gcc build-essential git clang libclang-dev
curl https://sh.rustup.rs -sSf | sh
rustup default stable
rustup toolchain install nightly-2021-01-20
rustup target add wasm32-unknown-unknown --toolchain nightly-2021-01-20
rustup update

Upgrading Rust Nightly Version

The current rust nightly version is fixed to the nightly build: nightly-2021-01-20 to ensure that the project always builds and the tests work. This is especially the case for the GitHub Actions pipelines which otherwise update their rust versions everytime they are triggered.

In order to upgrade the nightly version, the following commands need to be run:

rustup check # check if there is a new version
rustup update # updates the rust version

Manual Setup

Find manual setup instructions at the Substrate Developer Hub.

#### MacOS M1 Setup Extra Steps

You will need to perform a few actions manually in order for the project to compile on a M1 Mac.

  • make sure the top-level Cargo.toml is adjusted for the patching of rust-rocksdb
  • patch rust-rocksdb manually, by cloning and overwritting it: https://github.com/hdevalence/rust-rocksdb
  • patch wasmtime-runtime/src/traphandlers.rs:169 -> change (*cx.uc_mcontext).__ss.__rip as *const u8 to (*cx.uc_mcontext).__ss.__pc as *const u8 locally. The file is located here: /Users/<your-name>/.cargo/registry/src/github.com-<some-hash>/wasmtime-runtime-0.19.0/src/traphandlers.rs

Build

Once the development environment is set up, build the node template. This command will build the Wasm and native code:

WASM_BUILD_TOOLCHAIN=nightly-2021-01-20 cargo build --release

Docker Container

The following section describes how to build the provotum-mixnet docker image and how to use the Github container registry.

Github Container Registry

To use the images built and hosted on the Github container registry (ghcr.io), you need to be logged in.

Expose a Github personal access token with package:read rights and expose it:

  • locally as an ENV variable export CR_PAT=***
  • as a Github secret such that it can be used inside a Github Action: ${{ secrets.CR_PAT }}

Login (Local)

echo $CR_PAT | docker login ghcr.io -u $GITHUB_USER --password-stdin

Build Image (Local)

The command needs to be execute from the parent folder of: /node and /crypto (in this case called: provotum-mixnet) since both folders are required inside the Docker context during the build.

~/.../provotum-mixnet: DOCKER_BUILDKIT=1 docker build . -f ./node/Dockerfile --tag provotum-dev .
Note.

Once the project has been published and is publicy available. The path requirement for the crypto crate inside node/pallets/mixnet/Cargo.toml can be replaced with a reference to the Github project in which the crypto crate is hosted.

Different Architectures

To build the project such that it can run on different underlying architectures (e.g., amd64 and arm64), the necessary cross-compilation and emulation libraries need to be installed. On Linux (Ubuntu 20.04), the following needs to be installed:

sudo apt install qemu binfmt-support qemu-user-static

The Docker image can then be built using Docker Buildx.

~/.../provotum-mixnet: DOCKER_BUILDKIT=1 docker buildx build --platform "linux/amd64,linux/arm64" -f ./node/Dockerfile --tag provotum-docker .

Note. This is not recommend because it takes a 1h+ to build the arm64 image on Linux. I recommend to build the images natively whenever possible.

Build Image (Github Action)

A Github Action workflow exists to build the provotum docker container and push it to the Github container registry. Have a look at the .github/workflows/build.yml for more details.

CI-built Images

The GitHub-hosted runner can only build for the following architectures: Windows, MacOS, Linux

Tests

Run the following command to execute all tests.

cargo +nightly-2021-01-20 test

Run the following command to execute the test of the pallet-mixnet crate:

cargo +nightly-2021-01-20 test -p pallet-mixnet --features runtime-benchmarks

Benchmarks

Navigate into the folder: node/node and run the following command to check that all benchmarks are working correctly. Note: This executes the tests.

cargo +nightly-2021-01-20 test -p pallet-mixnet --features runtime-benchmarks

Build the provotum node including the benchmark feature by running the following command:

cargo +nightly-2021-01-20 build --features runtime-benchmarks

Note: To produce results that closely resemble the production environment, make sure to use the flag --release. Please, be aware that this will increase the compilation time. Also, the binary will end up inside ./target/release and not ./target/debug.

Navigate back to the folder: node.

  1. To list all existing commands of the pallet-mixnet crate run the following command:
./target/debug/provotum benchmark --chain dev --pallet "pallet_mixnet" --extrinsic "*" --repeat 0
  1. To perform all benchmarks run the following command. Note: The number of times each benchmark is executed can be changed via --repeat.
./target/debug/provotum benchmark --chain dev --pallet "pallet_mixnet" --extrinsic "*" --repeat 10

Run

Single Node Development Chain

Purge any existing dev chain state:

./target/release/provotum purge-chain --dev

Start a dev chain:

./target/release/provotum --dev

Or, start a dev chain with detailed logging:

RUST_LOG=debug RUST_BACKTRACE=1 ./target/release/provotum -lruntime=debug --dev

Multi-Node Local Testnet

To start a multi-node local test network, the docker-compose.yml file can be used.

docker-compose up

This starts a three-node local test network with:

  • Alice, as voting-authority (cannot author blocks, but is the voting admin)
  • Bob and Charlie, as sealers and PoA-authorities (can author blocks)

Network Modes

There are two possible network modes:

  • bridge all containers run in a separate docker network (e.g., 172.31.0.0/16, Alice on 172.31.0.2, Bob on 172.31.0.3, and so on...)
  • host all containers are exposed on the local network (e.g., 127.0.0.1, Alice on 127.0.0.1:9944, Bob on 127.0.0.1:9945, and so on...)

Note: If you want to curl one of the conatiners from the local network and the containers are running in bridge mode, it won't work. You need to either execute the curl from inside of one of the containers OR alterantively switch to the host network.

Switching networks can be done by commenting out the respective network block in the docker-compose.yml:

  • to activate host network mode: 
    alice:
  container_name: alice
  image: ghcr.io/meck93/provotum-mixnet:latest
  command: --chain=local --name Alice --base-path /tmp/alice --port 30333 --ws-port 9944 --rpc-port 9933
  network_mode: host
  # network_mode: bridge
  # ports:
  #   - 9944:9944 # (host_port:container_port)
  #   - 30333:30333
  #   - 9933:9933
  • to activate bridge network mode: 
    alice:
  container_name: alice
  image: ghcr.io/meck93/provotum-mixnet:latest
  command: --chain=local --name Alice --base-path /tmp/alice --port 30333 --ws-port 9944 --rpc-port 9933
  # network_mode: host
  network_mode: bridge
  ports:
    - 9944:9944 # (host_port:container_port)
    - 30333:30333
    - 9933:9933

Structure

A Substrate project such as this consists of a number of components that are spread across a few directories.

Node

A blockchain node is an application that allows users to participate in a blockchain network. Substrate-based blockchain nodes expose a number of capabilities:

  • Networking: Substrate nodes use the libp2p networking stack to allow the nodes in the network to communicate with one another.
  • Consensus: Blockchains must have a way to come to consensus on the state of the network. Substrate makes it possible to supply custom consensus engines and also ships with several consensus mechanisms that have been built on top of Web3 Foundation research.
  • RPC Server: A remote procedure call (RPC) server is used to interact with Substrate nodes.

There are several files in the node directory - take special note of the following:

  • chain_spec.rs: A chain specification is a source code file that defines a Substrate chain's initial (genesis) state. Chain specifications are useful for development and testing, and critical when architecting the launch of a production chain. Take note of the development_config and testnet_genesis functions, which are used to define the genesis state for the local development chain configuration. These functions identify some well-known accounts and use them to configure the blockchain's initial state.
  • service.rs: This file defines the node implementation. Take note of the libraries that this file imports and the names of the functions it invokes. In particular, there are references to consensus-related topics, such as the longest chain rule, the Aura block authoring mechanism and the GRANDPA finality gadget.

After the node has been built, refer to the embedded documentation to learn more about the capabilities and configuration parameters that it exposes:

./target/release/provotum --help

Runtime

In Substrate, the terms "runtime" and "state transition function" are analogous - they refer to the core logic of the blockchain that is responsible for validating blocks and executing the state changes they define. The Substrate project in this repository uses the FRAME framework to construct a blockchain runtime. FRAME allows runtime developers to declare domain-specific logic in modules called "pallets". At the heart of FRAME is a helpful macro language that makes it easy to create pallets and flexibly compose them to create blockchains that can address a variety of needs.

Review the FRAME runtime implementation included in this template and note the following:

  • This file configures several pallets to include in the runtime. Each pallet configuration is defined by a code block that begins with impl $PALLET_NAME::Trait for Runtime.
  • The pallets are composed into a single runtime by way of the construct_runtime! macro, which is part of the core FRAME Support library.

Pallets

The runtime in this project is constructed using many FRAME pallets that ship with the core Substrate repository and a template pallet that is defined in the pallets directory.

A FRAME pallet is compromised of a number of blockchain primitives:

  • Storage: FRAME defines a rich set of powerful storage abstractions that makes it easy to use Substrate's efficient key-value database to manage the evolving state of a blockchain.
  • Dispatchables: FRAME pallets define special types of functions that can be invoked (dispatched) from outside of the runtime in order to update its state.
  • Events: Substrate uses events to notify users of important changes in the runtime.
  • Errors: When a dispatchable fails, it returns an error.
  • Trait: The Trait configuration interface is used to define the types and parameters upon which a FRAME pallet depends.