sui-rpc-loadgen: Load Generator for SUI RPC Servers

sui-rpc-loadgen is a utility that facilitates the generation of read and write loads on single or multiple Sui RPC servers. Its primary functions include performance testing and data correctness verification.

Features

• Easily extendable to support any read/write endpoint
• Concurrent load generation with multiple threads, making it suitable for load testing high-traffic RPC servers.
• Cross-verifying results across multiple RPC Servers, ensuring data consistency and accuracy.
• Performance comparison between vanilla Full node RPC and Enhanced Full node RPC

Getting Started

Run the following command to see available commands:

cargo run --bin sui-rpc-loadgen -- -h

To try this locally, refer to the docs. Recommend setting database-url to an env variable. Note: run RUST_LOG="consensus=off" cargo run sui -- start --with-faucet --force-regenesis --with-indexer to rebuild.

Example 1: Get All Checkpoints

The following command initiates a single thread (num-threads == 1) to retrieve all checkpoints from the beginning (sequence 0) to the latest, executing the operation exactly once (repeat == 0):

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9124" --num-threads 1 get-checkpoints --start 0 --repeat 0 --interval-in-ms 0

This command is equivalent to the simplified version below:

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9124" --num-threads 1 get-checkpoints

Both commands achieve the same outcome: fetching all checkpoints using one thread, without repeating the operation.

By default, this command also verify all the transactions in the checkpoint, specify --skip-verify-transactions to disable fetching transactions. Note that this must used with --skip-verify-objects as we do need to fetch transactions to get objects for the checkpoint.

Note you must put --num-threads after the urls, otherwise the command will not be parsed correctly

Example 2: (WIP) Execute PaySui Transaction

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" --num-threads 1 pay-sui --repeat 100

NOTE: right now pay-sui only supports 1 thread but multi-threading support can be added pretty easily by assigning different gas coins to different threads

Example 3: Query Transaction Blocks

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 4 query-transaction-blocks --address-type from

Multi Get Transaction Blocks

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 4 multi-get-transaction-blocks

Multi Get Objects

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 4 multi-get-objects

Get Object

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 2 get-object --chunk-size 20

Get All Balances

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 2 get-all-balances --chunk-size 20

Get Reference Gas Price

cargo run --bin sui-rpc-loadgen -- --urls "http://127.0.0.1:9000" "http://127.0.0.1:9000" --num-threads 2 get-reference-gas-price --num-chunks-per-thread 10

Useful commands

cat sui-rpc-loadgen.b844f547-d354-4871-b958-1ea3fe23a0a8.log.2023-03-23 | awk '/Finished processing/{print $7}' | sort -n | uniq | awk 'BEGIN{last=0}{for(i=last+1;i<$1;i++) print i; last=$1} END{print last}' | tee missing_numbers.txt && wc -l missing_numbers.txt

Checks which checkpoints among threads have not been processed yet. The last one should be the largest checkpoint being processed.

wc -l missing_numbers.txt - counts how many checkpoints to go