Skip to content

Commit

Permalink
hack/benchmark: change boom to hey
Browse files Browse the repository at this point in the history
boom has moved to hey, due to a conflicting binary name with another
project
  • Loading branch information
Joshua Deare committed Oct 24, 2016
1 parent 0bb2384 commit 4eefdaa
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 20 deletions.
4 changes: 2 additions & 2 deletions Documentation/benchmarks/etcd-2-1-0-alpha-benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ GCE n1-highcpu-2 machine type

## Testing

Bootstrap another machine and use the [boom HTTP benchmark tool][boom] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.
Bootstrap another machine and use the [hey HTTP benchmark tool][hey] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.

## Performance

Expand Down Expand Up @@ -48,5 +48,5 @@ Bootstrap another machine and use the [boom HTTP benchmark tool][boom] to send r
| 256 | 64 | all servers | 1033 | 121.5 |
| 256 | 256 | all servers | 3061 | 119.3 |

[boom]: https://github.com/rakyll/boom
[hey]: https://github.com/rakyll/hey
[hack-benchmark]: /hack/benchmark/
4 changes: 2 additions & 2 deletions Documentation/benchmarks/etcd-2-2-0-benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Go OS/Arch: linux/amd64

## Testing

Bootstrap another machine, outside of the etcd cluster, and run the [`boom` HTTP benchmark tool](https://github.com/rakyll/boom) with a connection reuse patch to send requests to each etcd cluster member. See the [benchmark instructions](../../hack/benchmark/) for the patch and the steps to reproduce our procedures.
Bootstrap another machine, outside of the etcd cluster, and run the [`hey` HTTP benchmark tool](https://github.com/rakyll/hey) with a connection reuse patch to send requests to each etcd cluster member. See the [benchmark instructions](../../hack/benchmark/) for the patch and the steps to reproduce our procedures.

The performance is calulated through results of 100 benchmark rounds.

Expand Down Expand Up @@ -66,4 +66,4 @@ The performance is calulated through results of 100 benchmark rounds.

- Write QPS to cluster leaders seems to be increased by a small margin. This is because the main loop and entry apply loops were decoupled in the etcd raft logic, eliminating several blocks between them.

- Write QPS to all members seems to be increased by a significant margin, because followers now receive the latest commit index sooner, and commit proposals more quickly.
- Write QPS to all members seems to be increased by a significant margin, because followers now receive the latest commit index sooner, and commit proposals more quickly.
4 changes: 2 additions & 2 deletions Documentation/benchmarks/etcd-2-2-0-rc-benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Also, we use 3 etcd 2.1.0 alpha-stage members to form cluster to get base perfor

## Testing

Bootstrap another machine and use the [boom HTTP benchmark tool][boom] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.
Bootstrap another machine and use the [hey HTTP benchmark tool][hey] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.

## Performance

Expand Down Expand Up @@ -66,7 +66,7 @@ Bootstrap another machine and use the [boom HTTP benchmark tool][boom] to send r

- write QPS to all servers is increased by 30~80% because follower could receive latest commit index earlier and commit proposals faster.

[boom]: https://github.com/rakyll/boom
[hey]: https://github.com/rakyll/hey
[c7146bd5]: https://github.com/coreos/etcd/commits/c7146bd5f2c73716091262edc638401bb8229144
[etcd-2.1-benchmark]: etcd-2-1-0-alpha-benchmarks.md
[hack-benchmark]: /hack/benchmark/
28 changes: 14 additions & 14 deletions hack/benchmark/bench.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,57 +9,57 @@ keyarray=( 64 256 )
for keysize in ${keyarray[@]}; do

echo write, 1 client, $keysize key size, to leader
./boom -m PUT -n 10 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 1 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -m PUT -n 10 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 1 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo write, 64 client, $keysize key size, to leader
./boom -m PUT -n 640 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 64 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -m PUT -n 640 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 64 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo write, 256 client, $keysize key size, to leader
./boom -m PUT -n 2560 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 256 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -m PUT -n 2560 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 256 -T application/x-www-form-urlencoded $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo write, 64 client, $keysize key size, to all servers
for i in ${servers[@]}; do
./boom -m PUT -n 210 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 21 -T application/x-www-form-urlencoded $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo &
./hey -m PUT -n 210 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 21 -T application/x-www-form-urlencoded $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo &
done
# wait for all booms to start running
# wait for all heys to start running
sleep 3
# wait for all booms to finish
for pid in $(pgrep 'boom'); do
# wait for all heys to finish
for pid in $(pgrep 'hey'); do
while kill -0 "$pid" 2> /dev/null; do
sleep 3
done
done

echo write, 256 client, $keysize key size, to all servers
for i in ${servers[@]}; do
./boom -m PUT -n 850 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 85 -T application/x-www-form-urlencoded $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo &
./hey -m PUT -n 850 -d value=`head -c $keysize < /dev/zero | tr '\0' '\141'` -c 85 -T application/x-www-form-urlencoded $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo &
done
sleep 3
for pid in $(pgrep 'boom'); do
for pid in $(pgrep 'hey'); do
while kill -0 "$pid" 2> /dev/null; do
sleep 3
done
done

echo read, 1 client, $keysize key size, to leader
./boom -n 100 -c 1 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -n 100 -c 1 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo read, 64 client, $keysize key size, to leader
./boom -n 6400 -c 64 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -n 6400 -c 64 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo read, 256 client, $keysize key size, to leader
./boom -n 25600 -c 256 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -n 25600 -c 256 $leader/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo

echo read, 64 client, $keysize key size, to all servers
# bench servers one by one, so it doesn't overload this benchmark machine
# It doesn't impact correctness because read request doesn't involve peer interaction.
for i in ${servers[@]}; do
./boom -n 21000 -c 21 $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -n 21000 -c 21 $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
done

echo read, 256 client, $keysize key size, to all servers
for i in ${servers[@]}; do
./boom -n 85000 -c 85 $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
./hey -n 85000 -c 85 $i/v2/keys/foo | grep -e "Requests/sec" -e "Latency" -e "90%" | tr "\n" "\t" | xargs echo
done

done

0 comments on commit 4eefdaa

Please sign in to comment.