Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Jul 30, 2024

• Add 2B v2 configuration

  Show description for a050416
  

  Gemma Team

  authored and

  texasmichelle

  committed
  a050416

  Copy full SHA for a050416

• Fixed duplicate BOS token in training input.

  Show description for 20cb3da
  

  Gemma Team

  authored and

  texasmichelle

  committed
  20cb3da

  Copy full SHA for 20cb3da

• Add Ravin & Michelle to authors.

  Show description for 4dbac72
  texasmichelle

  committed
  4dbac72

  Copy full SHA for 4dbac72

• Remove the jax_numpy_rank_promotion config override from tests.

  Show description for ebf259b
  texasmichelle

  committed
  ebf259b

  Copy full SHA for ebf259b

• Update README to include a link to the v2 technical report.

  Show description for fc541d8
  texasmichelle

  committed
  fc541d8

  Copy full SHA for fc541d8

• Fix bug from reshape to transpose

  Show description for 5e2ff66
  

  Gemma Team

  authored and

  texasmichelle

  committed
  5e2ff66

  Copy full SHA for 5e2ff66

• Replace reciprocal computation in RMSNorm

  Show description for 1661656
  

  Gemma Team

  authored and

  texasmichelle

  committed
  1661656

  Copy full SHA for 1661656

• Add more query scaling options.

  Show description for a18d909
  

  Gemma Team

  authored and

  texasmichelle

  committed
  a18d909

  Copy full SHA for a18d909

• Ensure TramnsformerConfig.gemma_Xb() can be used as a constructors without relying on TransformerConfig.num_layers being pre-populated.

  Show description for 78e11e0
  

  Gemma Team

  authored and

  texasmichelle

  committed
  78e11e0

  Copy full SHA for 78e11e0

• Simplify ffw tests

  Show description for 335cb4a
  

  Gemma Team

  authored and

  texasmichelle

  committed
  335cb4a

  Copy full SHA for 335cb4a

• Add query-key and attention-value einsums.

  Show description for e6e2fee
  texasmichelle

  committed
  e6e2fee

  Copy full SHA for e6e2fee

• Add docstrings to clarify projections

  Show description for 621d651
  

  Gemma Team

  authored and

  texasmichelle

  committed
  621d651

  Copy full SHA for 621d651

• Simplify Gemma parameter loading

  Show description for 09cc659
  

  Gemma Team

  authored and

  texasmichelle

  committed
  09cc659

  Copy full SHA for 09cc659

• Fix bug in post ffw norm test, where norm was not being tested

  Show description for 7279d94
  

  Gemma Team

  authored and

  texasmichelle

  committed
  7279d94

  Copy full SHA for 7279d94

• Add query scaling options.

  Show description for acde1fc
  texasmichelle

  committed
  acde1fc

  Copy full SHA for acde1fc

• Move post-attention norm prior to the residual connection.

  Show description for 205e096
  texasmichelle

  committed
  205e096

  Copy full SHA for 205e096

• Improve readability of modules tests.

  Show description for 87c17fc
  texasmichelle

  committed
  87c17fc

  Copy full SHA for 87c17fc

• Fix logits capping for Gemma v2 27b and 9b

  Show description for ffe7bc1
  

  Gemma Team

  authored and

  texasmichelle

  committed
  ffe7bc1

  Copy full SHA for ffe7bc1

• Setting use_post_attn_norm=False and use_post_ffw_norm=False for 2b and 7b models.

  Show description for 295fd4a
  

  JasperSnoek

  authored and

  texasmichelle

  committed
  295fd4a

  Copy full SHA for 295fd4a

• Change default arguments to match types

  Show description for 12e9ff2
  

  Gemma Team

  authored and

  texasmichelle

  committed
  12e9ff2

  Copy full SHA for 12e9ff2

• Fix type hints to match usage

  Show description for 6628e36
  

  Gemma Team

  authored and

  texasmichelle

  committed
  6628e36

  Copy full SHA for 6628e36

• Add attention soft cap

  Show description for cd8d3f3
  

  Gemma Team

  authored and

  texasmichelle

  committed
  cd8d3f3

  Copy full SHA for cd8d3f3

• Add post-ffw norm to Gemma 27B flax implementation.

  Show description for 21b6b8b
  

  kkenealy

  authored and

  texasmichelle

  committed
  21b6b8b

  Copy full SHA for 21b6b8b

• Make attention types hashable tuples.

  Show description for 02c2335
  

  jrruijli

  authored and

  texasmichelle

  committed
  02c2335

  Copy full SHA for 02c2335

• Add logic for alternating new attention types

  Show description for bd559a7
  

  Gemma Team

  authored and

  texasmichelle

  committed
  bd559a7

  Copy full SHA for bd559a7

• Add post attention norm

  Show description for e125c5f
  

  Gemma Team

  authored and

  texasmichelle

  committed
  e125c5f

  Copy full SHA for e125c5f

• Add final logits softcap to transformer configuration

  Show description for 252ba76
  

  Gemma Team

  authored and

  texasmichelle

  committed
  252ba76

  Copy full SHA for 252ba76

• Add configurations for v2 models

  Show description for b0feead
  

  Gemma Team

  authored and

  texasmichelle

  committed
  b0feead

  Copy full SHA for b0feead

• Add sliding window attention to public gemma repository.

  Show description for d3696f0
  

  Shreya Pathak

  authored and

  texasmichelle

  committed
  d3696f0

  Copy full SHA for d3696f0

• Minor readability updates

  Show description for 59ffafe
  

  Gemma Team

  authored and

  texasmichelle

  committed
  59ffafe

  Copy full SHA for 59ffafe

• Fix bug where input mask is not correctly applied when max cache length is *shorter* than total sampling steps

  Show description for 9f197fb
  

  Gemma Team

  authored and

  texasmichelle

  committed
  9f197fb

  Copy full SHA for 9f197fb

• Fix bug due to the DecodeIds method of SentencePieceProcessor not terminating at the first eos_id. We thus mask all tokens after the first eos_id in the token_buffer to be pad_id.

  Show description for d609405
  

  Gemma Team

  authored and

  texasmichelle

  committed
  d609405

  Copy full SHA for d609405

• Fix a bug where positions do not increment after the last input token. This is because the input mask used for computing the positions was previously masking away the default padding tokens that fo…

  Show description for 95475a9
  

  Gemma Team

  authored and

  texasmichelle

  committed
  95475a9

  Copy full SHA for 95475a9

• Do not repeat the key/value projection

  Show description for 33c96b2
  

  Gemma Team

  authored and

  texasmichelle

  committed
  33c96b2

  Copy full SHA for 33c96b2

• Fix bug where sampler does not mask out padding tokens in decoding steps.

  Show description for bf2d158
  

  Gemma Team

  authored and

  texasmichelle

  committed
  bf2d158

  Copy full SHA for bf2d158

Pagination

Previous

Next