[2024.3] Our basic framework for downstream is based on SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition, Many thanks to this work for providing a strong baseline.
The main code for task specific pre-training is open sourced. Unfortunately, our code for the prototype decomposition part has been cleaned up by others. You can reproduce it based on the detailed description of our paper or contact with me for guidence. We are willing to offer help for reproduction.
This repository contains the open-sourced official implementation of the paper:
A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER (CIKM 2023 Oral Presentation).
If you find this repo helpful, please cite the following paper:
@inproceedings{dong2023multi,
title={A multi-task semantic decomposition framework with task-specific pre-training for few-shot ner},
author={Dong, Guanting and Wang, Zechen and Zhao, Jinxu and Zhao, Gang and Guo, Daichi and Fu, Dayuan and Hui, Tingfeng and Zeng, Chen and He, Keqing and Li, Xuefeng and others},
booktitle={Proceedings of the 32nd ACM International Conference on Information and Knowledge Management},
pages={430--440},
year={2023}
}we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.
