Skip to content

Commit

Permalink
update doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@paulbaogang
paulbaogang committed Nov 16, 2021
1 parent 0d58b0d commit 0232d21
Show file tree
Hide file tree
Showing 2 changed files with 109 additions and 13 deletions.
23 changes: 19 additions & 4 deletions ...oy/cluster-deploy/doc/fate_on_eggroll/Fate-allinone_deployment_guide_install.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ echo '/data/swapfile128G swap swap defaults 0 0' >> /etc/fstab
Or create by using the code package script in Section 5.1, and execute as app user:

```
sh /data/projects/fate-cluster-install/tools/makeVirtualDisk.sh
sh /data/projects/fate-cluster-install/tools-install/makeVirtualDisk.sh
Waring: please make sure has enough space of your disk first!!! (Please make sure there is enough storage space)
current user has sudo privilege(yes|no):yes (Whether the user has sudo privilege; enter yes and do not abbreviate it)
Enter store directory:/data (Set the storage path for virtual memory files; make sure the directory exists and do not set it to the root directory)
Expand Down Expand Up @@ -295,6 +295,7 @@ Description of Profile setup.conf
| roles | Default: "host", "guest" | The deployment roles: host or guest |
| version | Default: {version} | The version of FATE |
| pbase | Default: /data/projects | The root directory of the project |
| pname | Default: fate | project name |
| lbase | Default: /data/logs | Keep the default value and do not modify |
| ssh\_user | Default: app | he user who connects the destination machine<br> by ssh,and the owner of the deployed file |
| ssh\_group | Default: apps | The group which the user who connects the <br>destination machine by ssh belongs to, and <br>the group which the deployed file belongs to |
Expand Down Expand Up @@ -331,6 +332,9 @@ roles=( "host" "guest" )
version="{version}"
#project base
pbase="/data/projects"
#project name
pname="fate"
#log directory
lbase="/data/logs"
Expand Down Expand Up @@ -396,6 +400,9 @@ roles=( "host" )
version="{version}"
#project base
pbase="/data/projects"
#project name
pname="fate"
#log directory
lbase="/data/logs"
Expand Down Expand Up @@ -489,7 +496,7 @@ tail -f ./logs/deploy-mysql-host.log (Print the deployment status of mysql at

2\) Fateflow Logs

/data/projects/fate/fate\_flow/logs
/data/projects/fate/fateflow/logs/fate_flow

3\) Fateboard Logs

Expand Down Expand Up @@ -617,7 +624,7 @@ sh ./bin/eggroll.sh clustermanager start/stop/status/restart

```
source /data/projects/fate/bin/init_env.sh
cd /data/projects/fate/fate_flow/bin
cd /data/projects/fate/fateflow/bin
sh service.sh start|stop|status|restart
```

Expand Down Expand Up @@ -675,6 +682,14 @@ netstat -tlnp | grep 8080
| Service| Log Path
|----------|----------
| eggroll| /data/projects/fate/eggroll/logs
| fate\_flow \& task log| /data/projects/fate/fateflow/logs/
| fate\_flow \& task log| /data/projects/fate/fateflow/logs
| fateboard| /data/projects/fate/fateboard/logs
| mysql| /data/logs/mysql/

| Service | Log Path |
| --------------------- | -------------------------------------------------- |
| eggroll | /data/projects/fate/eggroll/logs |
| fate\_flow & task log | /data/projects/fate/fateflow/logs |
| fateboard | /data/projects/fate/fateboard/logs |
| mysql | /data/projects/fate/common/mysql/mysql-8.0.13/logs |

99 changes: 90 additions & 9 deletions ...cluster-deploy/doc/fate_on_eggroll/Fate-allinone_deployment_guide_install_zh.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ echo '/data/swapfile128G swap swap defaults 0 0' >> /etc/fstab
或者使用5.1章节的代码包中的脚本创建，app用户执行：

```
sh /data/projects/fate-cluster-install/tools/makeVirtualDisk.sh
sh /data/projects/fate-cluster-install/tools-install/makeVirtualDisk.sh
Waring: please make sure has enough space of your disk first!!! （请确认有足够的存储空间）
current user has sudo privilege(yes|no):yes （是否有sudo权限，输入yes，不能简写）
Enter store directory:/data （设置虚拟内存文件的存放路径，确保目录存在和不要设置在根目录）
Expand Down Expand Up @@ -307,6 +307,7 @@ vi fate-cluster-install/allInone/conf/setup.conf
| roles | 默认："host" "guest" | 部署的角色，有HOST端、GUEST端 |
| version | 默认：${version} | Fate 版本号 |
| pbase | 默认： /data/projects | 项目根目录 |
| pname | 默认：fate | 项目名称 |
| lbase | 默认：/data/logs | 保持默认不要修改 |
| ssh_user | 默认：app | ssh连接目标机器的用户，也是部署后文件的属主 |
| ssh_group | 默认：apps | ssh连接目标的用户的属组，也是部署后文件的属组 |
Expand Down Expand Up @@ -343,6 +344,9 @@ roles=( "host" "guest" )
version="${version}"
#project base
pbase="/data/projects"
#project name
pname="fate"
#log directory
lbase="/data/logs"
Expand Down Expand Up @@ -408,6 +412,9 @@ roles=( "host" )
version="${version}"
#project base
pbase="/data/projects"
#project name
pname="fate"
#log directory
lbase="/data/logs"
Expand Down Expand Up @@ -502,7 +509,7 @@ tail -f ./logs/deploy-mysql-host.log （实时打印HOST端mysql的部署情

2）fateflow日志

/data/projects/fate/fate_flow/logs/fate_flow/
/data/projects/fate/fateflow/logs/fate_flow

3）fateboard日志

Expand Down Expand Up @@ -636,7 +643,7 @@ sh ./bin/eggroll.sh clustermanager start/stop/status/restart

```
source /data/projects/fate/bin/init_env.sh
cd /data/projects/fate/python/fate_flow
cd /data/projects/fate/fateflow/bin
sh service.sh start|stop|status|restart
```

Expand Down Expand Up @@ -693,11 +700,85 @@ netstat -tlnp | grep 8080

## 7.3 服务日志

| 服务 | 日志路径 |
| ------------------ | ---------------------------------- |
| eggroll | /data/projects/fate/eggroll/logs |
| fate_flow&任务日志 | /data/projects/fate/python/logs |
| fateboard | /data/projects/fate/fateboard/logs |
| mysql | /data/logs/mysql/ |
| 服务 | 日志路径 |
| ------------------ | -------------------------------------------------- |
| eggroll | /data/projects/fate/eggroll/logs |
| fate_flow&任务日志 | /data/projects/fate/fateflow/logs |
| fateboard | /data/projects/fate/fateboard/logs |
| mysql | /data/projects/fate/common/mysql/mysql-8.0.13/logs |

## 7.4 空间清理规则

### 7.4.1 fateflow作业日志

所在机器：fate flow服务所在机器

目录：/data/projects/fate/fateflow/logs

保留期限：N=14天

规则：目录以$jobid开头，清理$jobid为N天前的数据

参考命令：   rm -rf /data/projects/fate/fateflow/logs/20211116*

### 7.4.2 fateflow系统日志

所在机器：fate flow服务所在机器

目录：/data/projects/fate/fateflow/logs/fate_flow

保留期限：N=14天

规则：以日期结尾，清理日期为N天前的数据

参考命令：   rm -rf /data/projects/fate/fateflow/logs/fate_flow/*.2021-11-16

### 7.4.3 EggRoll Session日志

所在机器：eggroll node节点

目录：/data/projects/fate/eggroll/logs/

保留期限：N=14天

规则：目录以$jobid开头，清理$jobid为N天前的数据

参考命令：   rm -rf /data/projects/fate/eggroll/logs/20211116*

### 7.4.3 EggRoll系统日志

所在机器：eggroll node节点

目录：/data/projects/fate/eggroll/logs/eggroll

保留期限：N=14天

规则：以日期结尾和以年份建立的历史文件夹中文件，清理N天前的数据

参考命令：   rm -rf /data/projects/fate/eggroll/logs/eggroll/\*.2021-11-16_*和

​ rm -rf /data/projects/fate/eggroll/logs/eggroll/2021/11/01

### 7.4.4 计算临时数据

所在机器：eggroll node节点

目录：/data/projects/fate/eggroll/data/IN_MEMORY

保留期限：N=7天

规则：namespace以$jobid开头，清理$jobid为N天前的数据

参考命令：   rm -rf /data/projects/fate/eggroll/data/IN_MEMORY/20211116*

### 7.4.5 作业组件输出数据

所在机器：eggroll node节点

目录：/data/projects/fate/eggroll/data/LMDB

保留期限：N=14天

规则：namespace以output_data_$jobid开头，清理$jobid为N天前的数据

参考命令：   rm -rf /data/projects/fate/eggroll/data/LMDB/output_data_20211116*

0 comments on commit 0232d21

Please sign in to comment.