Commits

Breadcrumbs

History for

on57c44b9

User selector

All users

Datepicker

All time

Commit History

Commits on Aug 31, 2023

• merge

  AleHD

  committed
  57c44b9

Commits on Aug 26, 2023

• added set argument of train_iters when loading checkpoint to make it easier to resume interrupted runs

  AleHD

  committed
  1d6a7d2

Commits on Aug 25, 2023

• added wandb_logger local save directory

  AleHD

  committed
  7eff638

• Add support for CodeLlama

  andreaskoepf

  committed
  db8dc4c

Commits on Aug 24, 2023

• Fixed --variable_seq_lengths

  Show description for 4409477
  

  AleHD

  and

  andreaskoepf

  committed
  4409477

• added --skip_iters option

  AleHD

  committed
  26fcedc

Commits on Aug 20, 2023

• removed roll

  Show description for fb1b35a
  

  AleHD

  and

  andreaskoepf

  committed
  fb1b35a

Commits on Aug 17, 2023

• Merge branch 'main' into instruction_tuning

  AleHD

  committed
  9d070fb

• Merge branch 'main' into testing

  AleHD

  committed
  dc218f3

• Merge branch 'main' into testing

  AleHD

  committed
  a893d22

• support for variable_seq length

  Show description for 3cef527
  

  AleHD

  and

  andreaskoepf

  committed
  3cef527

Commits on Aug 16, 2023

• adapted finetuning to support instruction data

  AleHD

  committed
  1228fc0

• Merge pull request epfLLM#26 from andreaskoepf/add_rope_scaling

  Show description for 950b01e
  mpagli

  authored
  950b01e

Commits on Aug 15, 2023

• check if grad_norm is finite

  

  andreaskoepf

  authored and

  Olivia-fsm

  committed
  5681ca9

Commits on Aug 13, 2023

• removed redundant assert not None check

  AleHD

  committed
  a230305

• fix typo

  andreaskoepf

  committed
  fd819dc

• Ensure freqs_cis tensor is on correct device

  andreaskoepf

  committed
  6aab9e7

• Add linear RoPE scaling & arbitary position_ids

  andreaskoepf

  committed
  8c7ea65

Commits on Aug 12, 2023

• Merge branch 'main' into add_lima_dropout

  AleHD

  authored
  7c68501

• Merge pull request epfLLM#20 from andreaskoepf/add-llama2-usage

  Show description for a3a2e0b
  AleHD

  authored
  a3a2e0b

Commits on Aug 10, 2023

• "does is not" -> "is not"

  andreaskoepf

  committed
  a6de710

• disable hidden_dropout warning when lima_dropout is specified

  andreaskoepf

  committed
  fac5016

• Add LIMA dropout

  andreaskoepf

  committed
  715136e

• "does uses" -> "uses"

  andreaskoepf

  authored
  90c2d0c

Commits on Aug 8, 2023

• Move flush_all() calls to _train()

  Show description for 6f30af2
  andreaskoepf

  authored
  6f30af2

Commits on Aug 3, 2023

• Merge pull request epfLLM#7 from epfLLM/optim

  Show description for 740ab3d
  AleHD

  authored
  740ab3d

Commits on Aug 1, 2023

• logging now occurs in every node

  AleHD

  committed
  8884236

• optimized positional embeddings

  AleHD

  committed
  c504227

Commits on Jul 26, 2023

• fixed wandb

  AleHD

  committed
  8592ba3

Commits on Jul 25, 2023

• removed redundant use_multiquery_attn option, just set num_attention_heads_kv=None to disable multiquery

  AleHD

  committed
  45a3fbf

Commits on Jul 24, 2023

• flash_attn 2.0 ready

  AleHD

  committed
  fa346be

• implemented falcon40b parallel layernorm

  AleHD

  committed
  d2c6b2e

Commits on Jul 23, 2023

• better way to save the position_embedding_type in the checkpoints

  AleHD

  committed
  396f3ec

• solved UserWarning: torch.distributed._reduce_scatter_base is a private function

  AleHD

  committed
  6f8c2b4

• added better explanation in the comments of the transformer layer forward

  AleHD

  committed
  6eb59d9

Pagination

Previous

Next