Skip to content

Latest commit

 

History

History

examples

README.rst

Example Usage Guide.

We provide here examples of FATE jobs, including FATE-Pipeline scripts, DSL conf files, and modeling quality comparison tasks

We suggest that user use example-runner tool FATE-Test.

Also, for the ease of submitting conf/dsl, we suggest that user install flow-client FATE-Client.

To quickly start model training and predictions using dsl & pipeline, you are welcomed to refer to：

  1. DSL v1 train and predict quick tutorial.
  2. DSL v2 train and predict quick tutorial.
  3. Pipeline train and predict quick tutorial.

Below lists included types of examples.

FATE-Pipeline

To enhance usability of FATE, starting at FATE-v1.5, FATE provides python APIs. User may develop federated learning models conveniently with FATE-Pipeline. We provide a host of Pipeline examples for each FATE module and a quick start guide for Pipeline here

Below code shows how to build and fit a hetero SecureBoost model with FATE-Pipeline in few lines.

import json
from pipeline.backend.config import Backend, WorkMode
from pipeline.backend.pipeline import PipeLine
from pipeline.component import Reader, DataIO, Intersection, HeteroSecureBoost, Evaluation
from pipeline.interface import Data
from pipeline.runtime.entity import JobParameters

guest_train_data = {"name": "breast_hetero_guest", "namespace": "experiment"}
host_train_data = {"name": "breast_hetero_host", "namespace": "experiment"}

# initialize pipeline
pipeline = PipeLine().set_initiator(role="guest", party_id=9999).set_roles(guest=9999, host=10000)

# define components
reader_0 = Reader(name="reader_0")
reader_0.get_party_instance(role="guest", party_id=9999).component_param(table=guest_train_data)
reader_0.get_party_instance(role="host", party_id=10000).component_param(table=host_train_data)
dataio_0 = DataIO(name="dataio_0", with_label=True)
dataio_0.get_party_instance(role="host", party_id=10000).component_param(with_label=False)
intersect_0 = Intersection(name="intersection_0")
hetero_secureboost_0 = HeteroSecureBoost(name="hetero_secureboost_0",
                                         num_trees=5,
                                         bin_num=16,
                                         task_type="classification",
                                         objective_param={"objective": "cross_entropy"},
                                         encrypt_param={"method": "iterativeAffine"},
                                         tree_param={"max_depth": 3})
evaluation_0 = Evaluation(name="evaluation_0", eval_type="binary")

# add components to pipeline, in order of task execution
pipeline.add_component(reader_0)\
    .add_component(dataio_0, data=Data(data=reader_0.output.data))\
    .add_component(intersect_0, data=Data(data=dataio_0.output.data))\
    .add_component(hetero_secureboost_0, data=Data(train_data=intersect_0.output.data))\
    .add_component(evaluation_0, data=Data(data=hetero_secureboost_0.output.data))

# compile & fit pipeline
pipeline.compile().fit(JobParameters(backend=Backend.EGGROLL, work_mode=WorkMode.STANDALONE))

# query component summary
print(f"Evaluation summary:\n{json.dumps(pipeline.get_component('evaluation_0').get_summary(), indent=4)}")

# Evaluation summary:
# {
#   "auc": 0.9971790603033666,
#   "ks": 0.9624094920987263
# }

Code for the above job can also be found here.

DSL

DSL is the first method FATE provides for constructing federated modeling jobs. For more information, please refer this guide on DSL.

Upgraded DSL(DSL v2) by FATE-v1.5 comes with the following major features:

  1. Predictions DSL may now be configured through FATE-Flow cli. Please note that new DSL no longer supports automatic formation of prediction DSL; user needs to first form DSL manually with FATE-Flow cli before running prediction task.
  2. New components may now be added to prediction DSL; for instance, evaluation module may be added to prediction task.
  3. Standardize style of role_parameter and algorithm_parameter.

For DSL v2 examples, please refer dsl/v2. For examples of the older version, please refer dsl/v1. This is the "federatedml-1.x-examples" in older version. Please note that starting at version 1.6, FATE may no longer support DSL v1.

Benchmark Quality

Starting at FATE-v1.5, FATE provides modeling quality verifier for comparing modeling quality of centralized training and FATE federated modeling. As of v1.5, we have provided quality comparison scripts for the following common models:

To run the comparison, please refer to the guide on benchmark_quality.

Upload Default Data

FATE provides a collection of publicly available data at data directory, along with a utility script for uploading all data sets. User may use the provided script to upload all pre-given data, or modify the corresponding configuration file for uploading arbitrary data. Please refer scripts for details.

Toy Example

FATE provides simple toy job for quick experiment when user developing FATE modules or testing for deployment. For details, please refer toy_example.

Min-test

Min-test is used for deployment testing and quick modeling demo. Min-test includes tasks of hetero Logistic Regression and hetero SecureBoost. User only needs to configure few simple parameters to run a full modeling job with FATE. Please refer min_test_task for instructions.