A pytorch implementation of SMART.
- python 3.7
- pytorch
- tqdm
- transformers
- sklearn
Model | CoLA(MCC) | QNLI(ACC) | SST-2(ACC) | MNLI(ACC) | QQP(ACC) | MRPC(ACC) | QNLI(ACC) |
---|---|---|---|---|---|---|---|
BERT-Base | 57.0 | 91.1 | 93.2 | 80.8 | 87.3 | 85.2 | 91.1 |
SMART-Base | 58.7 | 91.4 | 93.0 | 79.0 | 86.4 | 86.1 | 91.3 |
Download and prepare data.(To be implemented, download GLUE to glue_data/)
Prepare dataset
python main.py --do_prepare
Train model with SMART
python main.py --do_train --num_epoch 12 --batch_size 32 --task cola
Train model with normal bert fine-tuning
python main.py --do_train --normal --num_epoch 12 --batch_size 32 --task cola
- Add a script to download glue dataset and extract them to glue_data/.
- Prepare more experiment results.