Skip to content

Latest commit

 

History

History
71 lines (56 loc) · 3.59 KB

README.md

File metadata and controls

71 lines (56 loc) · 3.59 KB

PLATO-KAG

PLATO-KAG: Unsupervised Knowledge-Grounded Conversation via Joint Modeling

Abstract

Large-scale conversation models are turning to leveraging external knowledge to improve the factual accuracy in response generation. Considering the infeasibility to annotate the external knowledge for large-scale dialogue corpora, it is desirable to learn the knowledge selection and response generation in an unsupervised manner.

We propose PLATO-KAG (Knowledge-Augmented Generation), an unsupervised learning approach for end-to-end knowledge-grounded conversation modeling. For each dialogue context, the top-k relevant knowledge elements are selected and then employed in knowledge-grounded response generation. The two components of knowledge selection and response generation are optimized jointly and effectively under a balanced objective.

Experimental results on two publicly available datasets validate the superiority of PLATO-KAG.

Requirements

Usage

We provide models and scripts for our experiments on two datasets:

Preparation

Prepare models and datasets.

cd $DATASET
bash ./prepare.sh

It downloads the pre-trained models and corresponding fine-tuned model to ./${DATASET}/models:

  • 24L_NSP: the pre-trained 24L dialogue evaluation model optimized with NSP and MLM loss, for initializing the knowledge selection module.
  • 24L_SU: the pre-trained 24L dialogue generation model optimized with NLL loss, for initializing the knowledge-grounded response generation module.
  • 24L_PLATO_KAG_WOW: the PLATO-KAG model fine-tuned with Wizard of Wikipedia (wow).
  • 24L_PLATO_KAG_HOLLE: the PLATO-KAG model fine-tuned with Holl-E (holle).

It also downloads the corresponding dataset to ./${DATASET}/data.

Training

Use pre-trained models to fine-tune PLATO-KAG on the corresponding dataset.

cd $DATASET
bash ./train.sh

After training, you can find the training log and checkpoints in ./${DATASET}/output.

Inference and Evaluation

Use fine-tuned model to infer and evaluate the test set.

cd $DATASET
bash ./infer.sh

After inference and evaluation, you can find the results of inference and evaluation score in ./${DATASET}/output.

Citation

Please cite the paper if you use PLATO-KAG in your work:

@inproceedings{huang-etal-2021-plato,
    title = "{PLATO}-{KAG}: Unsupervised Knowledge-Grounded Conversation via Joint Modeling",
    author = "Huang, Xinxian and He, Huang and Bao, Siqi and Wang, Fan and Wu, Hua and Wang, Haifeng",
    booktitle = "Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI",
    month = nov,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.nlp4convai-1.14",
    doi = "10.18653/v1/2021.nlp4convai-1.14",
    pages = "143--154"
}

Contact Information

For help or issues using PLATO-KAG, please submit a GitHub issue.