Skip to content

Commit

Permalink
Add validation support
Browse files Browse the repository at this point in the history
* validation support (random split from testset)

* remove tensorboardX, use logger instead

* add random seed support
  • Loading branch information
songyouwei committed May 17, 2019
1 parent 0995ca8 commit 1139cff
Show file tree
Hide file tree
Showing 3 changed files with 94 additions and 325 deletions.
15 changes: 4 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@

* pytorch >= 0.4.0
* numpy >= 1.13.3
* tensorboardX >= 1.2
* python 3.6 / 3.7
* GloVe pre-trained word vectors (See [data_utils.py](./data_utils.py) for more detail)
* Download pre-trained word vectors [here](https://github.com/stanfordnlp/GloVe#download-pre-trained-word-vectors),
Expand All @@ -25,30 +24,24 @@
### Training

```sh
python train.py --model_name bert_spc --dataset restaurant --logdir bert_spc_logs
python train.py --model_name bert_spc --dataset restaurant
```

See [train.py](./train.py) for more detail.

#### See the training process (needs to install TensorFlow)

```sh
tensorboard --logdir=./bert_spc_logs
```
See [train.py](./train.py) for more training arguments.

### Inference

Please refer to [infer_example.py](./infer_example.py).

### Tips

* For non-BERT-based models, training procedure is not very stable.
* BERT-based models are more sensitive to hyperparameters (especially learning rate) on small data sets, see [this issue](https://github.com/songyouwei/ABSA-PyTorch/issues/27).
* Fine-tuning on the specific task is necessary for releasing the true power of BERT.
* Non-RNN models squeezed with [squeeze_embedding.py](./layers/squeeze_embedding.py) can be trained with larger batch size.

## BERT-based models

### AEN / AEN-BERT ([aen.py](./models/aen.py))
### AEN-BERT ([aen.py](./models/aen.py))
Song, Youwei, et al. "Attentional Encoder Network for Targeted Sentiment Classification." arXiv preprint arXiv:1902.09314 (2019). [[pdf]](https://arxiv.org/pdf/1902.09314.pdf)

![aen](assets/aen.png)
Expand Down
253 changes: 0 additions & 253 deletions cross_val.py

This file was deleted.

Loading

0 comments on commit 1139cff

Please sign in to comment.