RoBERTa embeddings are no longer a type of BERT embeddings (allenai#4771

)

* RoBERTa embeddings are no longer a type of BERT embeddings

* Changelog

CHANGELOG.md

@@ -19,6 +19,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   Previously, we would compute gradients from the top of the transformer, after aggregation from
   wordpieces to tokens, which gives results that are not very informative.  Now, we compute gradients
   with respect to the embedding layer, and aggregate wordpieces to tokens separately.
+- Fixed the heuristics for finding embedding layers in the case of RoBERTa. An update in the
+  `transformers` library broke our old heuristic.
 
 
 ## [v1.2.0](https://github.com/allenai/allennlp/releases/tag/v1.2.0) - 2020-10-29

allennlp/nn/util.py

@@ -1772,6 +1772,7 @@ def find_embedding_layer(model: torch.nn.Module) -> torch.nn.Module:
     from transformers.modeling_gpt2 import GPT2Model
     from transformers.modeling_bert import BertEmbeddings
     from transformers.modeling_albert import AlbertEmbeddings
+    from transformers.modeling_roberta import RobertaEmbeddings
     from allennlp.modules.text_field_embedders.text_field_embedder import TextFieldEmbedder
     from allennlp.modules.text_field_embedders.basic_text_field_embedder import (
         BasicTextFieldEmbedder,
@@ -1781,6 +1782,8 @@ def find_embedding_layer(model: torch.nn.Module) -> torch.nn.Module:
     for module in model.modules():
         if isinstance(module, BertEmbeddings):
             return module.word_embeddings
+        if isinstance(module, RobertaEmbeddings):
+            return module.word_embeddings
         if isinstance(module, AlbertEmbeddings):
             return module.word_embeddings
         if isinstance(module, GPT2Model):