Resolving Comparative Anaphora with and without Lexical Heads

Automatic anaphora resolution is an essential but challenging topic in the area of Natural Language Processing. It is crucial for natural language understanding systems such as discourse analysis, summarization, question answering, etc. Distinguishing from ordinary anaphora res olution, this work focuses on comparative anaphora resolution. "Comparative anaphora are referential noun phrases with non-pronominal heads which are introduced by lexical modifiers, such as other, another, similar or comparative adjectives, such as better, greater." [Klowersa 2021, p. 2]. The main task of this thesis is interpreting the relationship between the anaphor and its antecedent, and finding the correct antecedent of a referring comparative expression. The expressions of comparative anaphora are divided into two parts according to two different types — anaphor with or without lexical heads, i.e. the comparative anaphora corpus is divided into the WithLex and the WithoutLex corpus. This work applies neural methods to resolve comparative anaphora. One model is a BiLSTM model, which takes ELMo or GloVe embed- dings as input of a BiLSTM network and learns the span representation of the anaphor and its candidates (abbreviated as the BiLSTM model). The models then rank the pairwise score of every anaphor and each of its potential antecedents to predict its correct antecedent. Besides, extra features are also added. Additionally, another simplified model is applied - The output of ELMo, i.e. ELMo embeddings, will be directly fed into an FFNN (abbreviated as the ELMo model). This model just omits the additional BiLSTM part of the BiLSTM model above, since ELMo itself contains a BiLSTM model. Because the comparative anaphora annotations are very limited, Transfer Learning is also applied. The ELMo model will be first pre-trained on the corpus I created for the pronoun resolution task. The model’s parameters are optimized by training on the source task, pronominal anaphora resolution, and transferred to the target task, comparative anaphora resolution. The first hypothesis of this thesis is that 1. there are sig- nificant differences between comparative anaphors with and without lexical heads. The second is that 2. resolving the antecedent of anaphor without lexical heads is extremely context-dependent. The third is that 3. including extra features will im- prove the models’ performance. The fourth is that 4. pre-training on pronominal anaphora data might improve the model performance for the anaphora resolution task on WithoutLex. The fifth is that 5. additionally concatenating Phrase-BERT embedding can improve the model performance.