Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Dec 16, 2024

• Update README.md

  neverix

  authored
  6ac21d2

  Copy full SHA for 6ac21d2

Commits on Sep 16, 2024

• Merge remote-tracking branch 'origin/main'

  neverix

  committed
  ad51837

  Copy full SHA for ad51837

• Update dependencies, fix other dumb things, train last layer SAE

  neverix

  committed
  a4502a7

  Copy full SHA for a4502a7

Commits on Aug 28, 2024

• Fix 8-bit gradient quantization

  neverix

  committed
  3d07bcc

  Copy full SHA for 3d07bcc

• Gradient quantization does not work

  neverix

  committed
  b3ab6b7

  Copy full SHA for b3ab6b7

Commits on Aug 22, 2024

• Hadamard gradient quantization

  neverix

  committed
  c2d8382

  Copy full SHA for c2d8382

Commits on Aug 19, 2024

• Trying f32->8 quantization

  neverix

  committed
  72de10b

  Copy full SHA for 72de10b

Commits on Aug 17, 2024

• Nice 8-bit f16 quantization

  neverix

  committed
  6280817

  Copy full SHA for 6280817

Commits on Aug 6, 2024

• Gemma 2 SAEs

  neverix

  committed
  4dc3ba5

  Copy full SHA for 4dc3ba5

Commits on Aug 5, 2024

• Finish training Gemma transcoders

  neverix

  committed
  f20fc24

  Copy full SHA for f20fc24

Commits on Aug 2, 2024

• Update

  neverix

  committed
  2e3dc93

  Copy full SHA for 2e3dc93

• Merge remote-tracking branch 'origin/main'

  neverix

  committed
  bac4ab6

  Copy full SHA for bac4ab6

• Quantization tests

  neverix

  committed
  91511c2

  Copy full SHA for 91511c2

• Configurable approximate topK

  neverix

  committed
  56400ff

  Copy full SHA for 56400ff

Commits on Jul 23, 2024

• Merge remote-tracking branch 'origin/main'

  neverix

  committed
  e67e0ef

  Copy full SHA for e67e0ef

• Retrain L12

  neverix

  committed
  211f610

  Copy full SHA for 211f610

• Start training new L12 SAE

  neverix

  committed
  2b3c8af

  Copy full SHA for 2b3c8af

• Fix SAE restoring

  neverix

  committed
  ffe3482

  Copy full SHA for ffe3482

• Fix transcoder training by tracking target norm

  neverix

  committed
  8e63dcc

  Copy full SHA for 8e63dcc

Commits on Jul 13, 2024

• Approximate top-K works well

  neverix

  committed
  bf3eb74

  Copy full SHA for bf3eb74

• Fix transcoder training

  neverix

  committed
  abf2e35

  Copy full SHA for abf2e35

• Quantize to 8-bit

  neverix

  committed
  a56052b

  Copy full SHA for a56052b

Commits on Jul 12, 2024

• Draft transcoders

  neverix

  committed
  734c190

  Copy full SHA for 734c190

Commits on Jul 10, 2024

• Train more Instruct v1 SAEs

  neverix

  committed
  639dad3

  Copy full SHA for 639dad3

Commits on Jul 9, 2024

• Top-K works with sparsity if you apply sparsity in a separate kernel

  neverix

  committed
  5b7dd84

  Copy full SHA for 5b7dd84

• AQT is slower (:goose10:)

  neverix

  committed
  1e34a6c

  Copy full SHA for 1e34a6c

• Adafactor can be improved

  neverix

  committed
  e1336c8

  Copy full SHA for e1336c8

• Implement Adadelta on old Optax

  neverix

  committed
  af59b62

  Copy full SHA for af59b62

• fixes; isort; 8-bit weights work, 8-bit Adam doesn't work

  neverix

  committed
  953cf8d

  Copy full SHA for 953cf8d

Commits on Jul 8, 2024

• Very smooth curves

  neverix

  committed
  72a3f1a

  Copy full SHA for 72a3f1a

• Restore training stability with this one weird trick (+fix Anthropic impl)

  neverix

  committed
  94bca45

  Copy full SHA for 94bca45

• Fail to fix training instabilities

  neverix

  committed
  ea61a29

  Copy full SHA for ea61a29

Commits on Jul 6, 2024

• Instruct v0 training

  neverix

  committed
  6513612

  Copy full SHA for 6513612

Commits on Jul 4, 2024

• Try to fix training instabilities

  neverix

  committed
  18fc20f

  Copy full SHA for 18fc20f

Commits on Jul 3, 2024

• Try (and fail) to fix numerical (?) issue

  neverix

  committed
  e4b42de

  Copy full SHA for e4b42de

Pagination

Previous

Next