Skip to content

murray-z/bert_of_theseus_torch

Repository files navigation

bert_theuse_torch

實現《BERT-of-Theseus: Compressing BERT by Progressive Module Replacing》, 由於沒有GPU,沒有測試。

使用

  • 配置文件config.py
  • 下载模型放在bert_config目录下
  • 數據預處理
    • python data_preprocess.py
  • 訓練模型
    • python train_theseus.py

requirements

  • transformers=4.18.0

About

BERT-of-Theseus 模型壓縮方法的pytorch實現

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages