Skip to content

Commit

Permalink
MISC - fixed typo when using multiple blocks in attentive probe
Browse files Browse the repository at this point in the history
  • Loading branch information
MidoAssran committed Feb 16, 2024
1 parent 0f772c5 commit 2dfdfac
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/models/attentive_pooler.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def rescale(param, layer_id):
else:
rescale(self.cross_attention_block.proj.weight.data, 1)
if self.blocks is not None:
for layer_id, layer in enumerate(1, self.blocks):
for layer_id, layer in enumerate(self.blocks, 1):
rescale(layer.attn.proj.weight.data, layer_id + 1)
rescale(layer.mlp.fc2.weight.data, layer_id + 1)

Expand Down

0 comments on commit 2dfdfac

Please sign in to comment.