Skip to content

Commit

Permalink
Merge pull request FederatedAI#3069 from FederatedAI/develop-1.6.1-do…
Browse files Browse the repository at this point in the history
…c_update

add guide
  • Loading branch information
zhihuiwan authored Sep 3, 2021
2 parents 4404222 + 71527bc commit 830353c
Show file tree
Hide file tree
Showing 9 changed files with 87 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[submodule "fateboard"]
path = fateboard
url = https://github.com/FederatedAI/FATE-Board.git
branch = develop-1.6.1
branch = v1.6.1
[submodule "eggroll"]
path = eggroll
url = https://github.com/WeBankFinTech/eggroll.git
Expand Down
2 changes: 2 additions & 0 deletions python/fate_flow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ Usage

##### [HTTP API](./doc/fate_flow_http_api.rst)

##### [DATE ACCESS](./doc/Data_Access_Guide_zh.md)

##### [Training Examples](../../examples/README.rst)

##### [Online Inference Examples](../../doc/model_publish_with_serving_guide.md)
Expand Down
61 changes: 61 additions & 0 deletions python/fate_flow/doc/Data_Access_Guide_zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
## FATE数据接入指南

### 1. upload

#### fate提供upload组件供用户上传数据至fate所支持的存储系统内(默认eggroll)
用法如下:
```shell script
# cli
python fate_flow_client.py -f upload -c $config_path

# fate client
flow data upload -c $config_path
```
- **eggroll**

```
python fate_flow_client.py -f upload -c examples/upload/upload_to_eggroll.json
```
```
flow data upload -c examples/upload/upload_to_eggroll.json
```
- **mysql**
```
python fate_flow_client.py -f upload -c examples/upload/upload_to_mysql.json
```
```
flow data upload -c examples/upload/upload_to_mysql.json
```
- **hdfs**
```
python fate_flow_client.py -f upload -c examples/upload/upload_to_hdfs.json
```
```
flow data upload -c examples/upload/upload_to_hdfs.json
```

### 2. table bind
#### 若用户的数据已经存在fate所支持的存储系统内,可通过table bind将真实存储路径映射到fate存储表
用法如下:
```shell script
# cli
python fate_flow_client.py -f table_bind -c $config_path

# fate client
flow table bind -c $config_path
```

- **mysql**
```
python fate_flow_client.py -f table_bind -c examples/table_bind/bind_mysql_table.json
```
```
flow table bind -c examples/table_bind/bind_mysql_table.json
```
- **hdfs**
```
python fate_flow_client.py -f table_bind -c examples/table_bind/bind_hdfs_table.json
```
```
flow table bind examples/table_bind/bind_hdfs_table.json
```
2 changes: 1 addition & 1 deletion python/fate_flow/doc/fate_flow_cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ table_bind

.. code:: bash
python fate_flow_client.py -f table_bind -c examples/bind_mysql_table.json
python fate_flow_client.py -f table_bind -c examples/table_bind/bind_hdfs_table.json
Model
Expand Down
11 changes: 11 additions & 0 deletions python/fate_flow/examples/upload/upload_to_eggroll.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"file": "examples/data/breast_hetero_guest.csv",
"id_delimiter": ",",
"head": 1,
"partition": 4,
"work_mode": 1,
"backend": 1,
"namespace": "experiment",
"table_name": "breast_hetero_guest",
"task_cores": 2
}
11 changes: 11 additions & 0 deletions python/fate_flow/examples/upload/upload_to_hdfs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"file": "examples/data/breast_hetero_guest.csv",
"id_delimiter": ",",
"head": 1,
"partition": 10,
"work_mode": 1,
"backend": 1,
"namespace": "experiment",
"table_name": "breast_hetero_guest",
"storage_engine": "HDFS"
}

0 comments on commit 830353c

Please sign in to comment.