4 Pull requests merged by 2 people

• Add k decay schedule

  #53 merged Feb 20, 2025

• Use rank 0 if not distributed

  #52 merged Feb 19, 2025

• Fix illegal memory access in Xformers kernel

  #51 merged Feb 17, 2025

• Fix resume - num_tokens_since_fired is rank dependent

  #50 merged Feb 14, 2025

1 Pull request opened by 1 person

• Switch to Schedule-Free Signum (Sign SGD) optimizer

  #54 opened Feb 20, 2025