TODO : need to record a new video here ...
You can download Pumba binary for your OS from release page.
$ pumba help
Pumba version v0.2.0
NAME:
Pumba - Pumba is a resilience testing tool, that helps applications tolerate random Docker container failures: process, network and performance.
USAGE:
pumba [global options] command [command options] containers (name, list of names, RE2 regex)
VERSION:
v0.2.0
COMMANDS:
kill kill specified containers
netem emulate the properties of wide area networks
pause pause all processes
stop stop containers
rm remove containers
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--host value, -H value daemon socket to connect to (default: "unix:///var/run/docker.sock") [$DOCKER_HOST]
--tls use TLS; implied by --tlsverify
--tlsverify use TLS and verify the remote [$DOCKER_TLS_VERIFY]
--tlscacert value trust certs signed only by this CA (default: "/etc/ssl/docker/ca.pem")
--tlscert value client certificate for TLS authentication (default: "/etc/ssl/docker/cert.pem")
--tlskey value client key for TLS authentication (default: "/etc/ssl/docker/key.pem")
--debug enable debug mode with verbose logging
--json produce log in JSON format: Logstash and Splunk friendly
--slackhook value web hook url; send Pumba log events to Slack
--slackchannel value Slack channel (default #pumba) (default: "#pumba")
--interval value, -i value recurrent interval for chaos command; use with optional unit suffix: 'ms/s/m/h'
--random, -r randomly select single matching container from list of target containers
--dry dry runl does not create chaos, only logs planned chaos commands
--help, -h show help
--version, -v print the version
$ pumba kill -h
NAME:
pumba kill - kill specified containers
USAGE:
pumba kill [command options] containers (name, list of names, RE2 regex)
DESCRIPTION:
send termination signal to the main process inside target container(s)
OPTIONS:
--signal value, -s value termination signal, that will be sent by Pumba to the main process inside target container(s) (default: "SIGKILL")
$ pumba pause -h
NAME:
pumba pause - pause all processes
USAGE:
pumba pause [command options] containers (name, list of names, RE2 regex)
DESCRIPTION:
pause all running processes within target containers
OPTIONS:
--duration value, -d value pause duration: should be smaller than recurrent interval; use with optional unit suffix: 'ms/s/m/h'
$ pumba stop -h
NAME:
pumba stop - stop containers
USAGE:
pumba stop [command options] containers (name, list of names, RE2 regex)
DESCRIPTION:
stop the main process inside target containers, sending SIGTERM, and then SIGKILL after a grace period
OPTIONS:
--time value, -t value seconds to wait for stop before killing container (default 10) (default: 10)
$ pumba rm -h
NAME:
pumba rm - remove containers
USAGE:
pumba rm [command options] containers (name, list of names, RE2 regex)
DESCRIPTION:
remove target containers, with links and voluems
OPTIONS:
--force, -f force the removal of a running container (with SIGKILL)
--links, -l remove container links
--volumes, -v remove volumes associated with the container
$ pumba netem -h
NAME:
Pumba netem - delay, loss, duplicate and re-order (run 'netem') packets, to emulate different network problems
USAGE:
Pumba netem command [command options] [arguments...]
COMMANDS:
delay dealy egress traffic
loss
duplicate
corrupt
OPTIONS:
--duration value, -d value network emulation duration; should be smaller than recurrent interval; use with optional unit suffix: 'ms/s/m/h'
--interface value, -i value network interface to apply delay on (default: "eth0")
--target value, -t value target IP filter; netem will impact only on traffic to target IP
--help, -h show help
NAME:
Pumba netem - delay, loss, duplicate and re-order (run 'netem') packets, to emulate different network problems
USAGE:
Pumba netem command [command options] [arguments...]
COMMANDS:
delay dealy egress traffic
loss
duplicate
corrupt
OPTIONS:
--duration value, -d value network emulation duration; should be smaller than recurrent interval; use with optional unit suffix: 'ms/s/m/h'
--interface value, -i value network interface to apply delay on (default: "eth0")
--target value, -t value target IP filter; netem will impact only on traffic to target IP
--help, -h show help
$ pumba netem delay -h
NAME:
Pumba netem delay - dealy egress traffic
USAGE:
Pumba netem delay [command options] containers (name, list of names, RE2 regex)
DESCRIPTION:
dealy egress traffic for specified containers; networks show variability so it is possible to add random variation; delay variation isn't purely random, so to emulate that there is a correlation
OPTIONS:
--amount value, -a value delay amount; in milliseconds (default: 100)
--variation value, -v value random delay variation; in milliseconds; example: 100ms ± 10ms (default: 10)
--correlation value, -c value delay correlation; in percents (default: 20)
$ pumba --debug --interval 5m --random netem --duration 2m --interface eth2 delay --amount 2000 re2:^result
Once in 5 minutes, Pumba will delay for 2 seconds (2000ms) egress traffic for some (randomly chosen) container named result... (matching ^result regexp) on eth2 network interface. Pumba will restore normal connectivity after 2 minutes.
If you choose to use Pumba Docker image on Linux, use the following command:
docker run -d -v /var/run/docker.sock:/var/run/docker.sock gaiaadm/pumba pumba kill --interval 10s --signal SIGTERM ^hp
The above command, once in a 10 seconds, tries to kill (with SIGTERM signal) all containers named hp(something) on same Docker host, where Pumba container is running.
Note: For Windows and OS X you will need to use --host argument, since there is no unix socket /var/run/docker.sock to mount.
If you are running Kubernetes >= 1.1.0. You can take advantage of DaemonSets to automatically deploy the Pumba on all your nodes. On 1.1.x you'll need to explicitly enable the DaemonSets extension, see http://kubernetes.io/v1.1/docs/admin/daemons.html#caveats.
You'll then be able to deploy the DaemonSet with the command
$ kubectl create -f pumba_kube.yml
If you are not running Kubernetes >= 1.1.0 or do not want to use DaemonSets, you can also run the Pumba as a regular docker container on each node you want to make chaos (see above)
If you are running CoreOS cluster. You can use fleetctl command to deploy Pumba service file on every CoreOS cluster node.
You'll then be able to deploy the Pumba service with the command
$ fleetctl start pumba_coreos.service
You can build Pumba with or without Go installed on your machine.
In order to build Pumba, you need to have Go 1.6+ setup on your machine.
Here is the approximate list of commands you will need to run:
cd $GOPATH
mkdir github.com/gaia-adm && cd github.com/gaia-adm
git clone [email protected]:gaia-adm/pumba.git
cd pumba
glide install
go build -v
You do not have to install and configure Go in order to build and test Pumba project. Pubma builder Docker image contains Go 1.6 and all tools required to build and test Pumba.
First of all clone Pumba git repository:
git clone [email protected]:gaia-adm/pumba.git
cd pumba
Now create a Pumba builder Docker image.
docker build -t pumba/builder -f Build.Dockerfile .
Now you can use pumba/builder to build, test (with coverage) and deploy Pumba.
To build a new Pumba binary run the following command:
docker run --rm -v "$PWD":/go/src/github.com/gaia-adm/pumba -w /go/src/github.com/gaia-adm/pumba pumba/builder script/go_build.sh
To build new Pumba binaries for multiple platforms run the following command (using gox tool):
docker run --rm -v "$PWD":/go/src/github.com/gaia-adm/pumba -w /go/src/github.com/gaia-adm/pumba pumba/builder script/gox_build.sh
To run all Pumba tests and generate coverage report run the following command:
docker run --rm -v "$PWD":/go/src/github.com/gaia-adm/pumba -w /go/src/github.com/gaia-adm/pumba pumba/builder script/coverage.sh --html
- Official Docker Engine API for Go docker/engine-api
- Docker Client samalba/dockerclient - refactoring to Docker Engine API
- Logging Sirupsen/logrus
- Command line app lib codegangsta/cli
I've also borrowed some code from very good CenturyLinkLabs/watchtower project.
Code is under the Apache License v2.