update doc (modelscope#1789)

zss205 · Aug 22, 2024 · 2e47ded · 2e47ded
1 parent b940c74
commit 2e47ded
Show file tree

Hide file tree

Showing 2 changed files with 9 additions and 0 deletions.
diff --git a/docs/source/LLM/命令行参数.md b/docs/source/LLM/命令行参数.md
@@ -250,6 +250,10 @@ IA3使用`target_modules`, `target_regex`, `modules_to_save`三个参数.
 
 以下参数`sft_type`设置为`reft`时生效.
 
+> 1. ReFT无法合并tuner
+> 2. ReFT和gradient_checkpointing不兼容
+> 3. 如果使用DeepSpeed遇到问题请暂时卸载DeepSpeed
+
 - `--reft_layers`: ReFT应用于哪些层上, 默认为`None`, 代表所有层, 可以输入层号的list, 例如`--reft_layers 1 2 3 4`
 - `--reft_rank`: ReFT矩阵的rank, 默认为`4`.
 - `--reft_intervention_type`: ReFT的类型, 支持'NoreftIntervention', 'LoreftIntervention', 'ConsreftIntervention', 'LobireftIntervention', 'DireftIntervention', 'NodireftIntervention', 默认为`LoreftIntervention`.

diff --git a/docs/source_en/LLM/Command-line-parameters.md b/docs/source_en/LLM/Command-line-parameters.md
@@ -251,6 +251,11 @@ The following parameters take effect when `sft_type` is set to `ia3`.
 ### ReFT Fine-tuning Parameters
 
 The following parameters take effect when the `sft_type` is set to `reft`.
+
+> 1. ReFT tuner cannot be merged
+> 2. ReFT and gradient_checkpointing are not compatible
+> 3. If error happens when using ReFT and DeepSpeed, please uninstall DeepSpeed
+
 - `--reft_layers`: Specifies which layers ReFT is applied to; defaults to `None`, meaning all layers. You can input a list of layer numbers, for example: `--reft_layers 1 2 3 4`.
 - `--reft_rank`: The rank of the ReFT matrix; defaults to `4`.
 - `--reft_intervention_type`: The type of ReFT intervention, supporting 'NoreftIntervention', 'LoreftIntervention', 'ConsreftIntervention', 'LobireftIntervention', 'DireftIntervention', and 'NodireftIntervention'; defaults to `LoreftIntervention`.