Skip to content

Commit

Permalink
quote
Browse files Browse the repository at this point in the history
  • Loading branch information
callummcdougall committed Nov 2, 2023
1 parent 846fde8 commit 649a090
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -3383,7 +3383,7 @@ def show_multiple(dataset: SortedListDataset):
## Summary of how the model works
In the second half of the sequence, the attention heads perform the algorithm "attend back to (and copy) the first token which is larger than me. For example, in a sequence like:
In the second half of the sequence, the attention heads perform the algorithm "attend back to (and copy) the first token which is larger than me". For example, in a sequence like:
```
[7, 5, 12, 3, SEP, 3, 5, 7, 12]
Expand Down

0 comments on commit 649a090

Please sign in to comment.