Based on fast.ai library, version 0.7.0.
-
training.ipynb- notebook with classifier of relevant-irrelevant articles, and multilabel classifier of manipulative texts, both in Ukrainian and Russian -
itos_<lang>.pkl- token dictionaries -
ru/,uk/- folders with fastai models. Put models in <lang>/models - links to pretrained LM and classifiers:- Wikipedia language model for Russian, forward LSTM
- Wikipedia language model for Ukrainian, forward LSTM
- Finetuned on news corpus Wikipedia language model for Russian
- Finetuned on news corpus Wikipedia language model for Ukrainian
- Finetuned on news corpus Wikipedia language model for Russian, encoder only
- Finetuned on news corpus Wikipedia language model for Ukrainian, encoder only
- Classifier of relevant news in Russian
- Classifier of relevant news in Ukrainian
- Classifier of types of manipulation for Russian
- Classifier of types of manipulation for Ukrainian
-
<lang>_test_set.jl- more thoroughly annotated random sets of news in Russian and Ukraininan, that were not involved in training.