Instructions of test tools

1.Use

Execute commands
cd examples/test
python run_test.py default_env.json

run_test.py search and execute tasks defined by users.
default_env.json environment configs based on users' running environment.

Optional parameters
"-o", "--output", "file to save result, defaults to test_result"
"-e", "--error", "file to save error"
"-i", "--interval", "check job status every i seconds, defaults to 3"
"--skip_data", "skip data upload, used to be false if not use

mutually_exclusive_group include:
"-d", "--dir", "dir to find testsuites",
"-s", "--suite","a single testsuite to run"

2.Tips

If '-d' or '-s' is not given,the script will execute the tasks defined in task files from examples/dsl/v1 folder with a "testsuite.json" suffix.
If there is a '-d' or '-s' parameter,the script will execute the tasks defined in task files with a "testsuite.json" suffix from the dir given by '-d' or a single task file given by '-s'. An example task file is given in examples/test/demo/temp_testsuite.json including a training and a prediction task.

3.Config files

default_env.json
Please set role id in "role", including host, guest, and arbiter.
Please build the relationship between roles and ip in "ip_map",where -1 stands for local,and remote host will be given ip address.

testsuite.json
You can submit data for many tasks once in "data",and each has a series of configs in a dict.
"role" parameter describes the location of the data defined in default_env.json.For example, "guest_0" represents the data located in the first guest defined in the guest list of default_env.json.
You can define your own tasks in "tasks".Training tasks and prediction tasks are supported now. There is some difference between them.
A prediction task needs to state the task name of the training task which it depends on.
Please name different tasks with different names,if two tasks share the same name,you will get the execution result of the letter defined.

demo:

python run_test.py default_env.json -s ./demo/temp_testsuite.json

4.Examples of results

./demo/temp_testsuite.json
====================================================================
lr	success	201912271619411350983
lr-predict	success	201912271620429623264