Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Jan 12, 2024

• Pin Phi 2 HF revision (Lightning-AI#875)

  carmocca

  authored
  0676f23

  Copy full SHA for 0676f23

• Mixtral supports a fully dense context length of 32k tokens (Lightning-AI#874)

  carmocca

  authored
  3fd3c9b

  Copy full SHA for 3fd3c9b

Commits on Jan 10, 2024

• Use fused AdamW in pretraining script (Lightning-AI#857)

  awaelchli

  authored
  436b318

  Copy full SHA for 436b318

• Refactor get_lr in pretraining scripts (Lightning-AI#854)

  awaelchli

  authored
  86c4fa7

  Copy full SHA for 86c4fa7

• Pin Phi HF revision (Lightning-AI#869)

  carmocca

  authored
  091111b

  Copy full SHA for 091111b

• Tensor-parallel generation on multiple devices (Lightning-AI#835)

  carmocca

  authored
  51d5328

  Copy full SHA for 51d5328

• Add reference to TinyLlama reproduction studio (Lightning-AI#867)

  awaelchli

  authored
  7dd9ff2

  Copy full SHA for 7dd9ff2

Commits on Jan 5, 2024

• Add License at the top of individual files (Lightning-AI#860)

  lantiga

  authored
  ad65549

  Copy full SHA for ad65549

• Add bibtex citation to README (Lightning-AI#859)

  awaelchli

  authored
  12e962d

  Copy full SHA for 12e962d

Commits on Dec 29, 2023

• Mixtral: LLaMAMoE wrongly inherited from base LLaMAMLP for Adapter_v2 and LoRA (Lightning-AI#851)

  Andrei-Aksionov

  authored
  cd1c521

  Copy full SHA for cd1c521

Commits on Dec 24, 2023

• Fix logic to count devices (Lightning-AI#846)

  marco-c

  authored
  5a8ec86

  Copy full SHA for 5a8ec86

Commits on Dec 21, 2023

• LoRA merge with quantization: optimiziations (Lightning-AI#842)

  Andrei-Aksionov

  authored
  70164f8

  Copy full SHA for 70164f8

• Sequential generation on multiple devices (Lightning-AI#815)

  carmocca

  authored
  c680a88

  Copy full SHA for c680a88

Commits on Dec 20, 2023

• Updates to the tinyllama pretrain script (Lightning-AI#843)

  awaelchli

  authored
  d36a698

  Copy full SHA for d36a698

• Updated GitHub markdown alerts (Lightning-AI#841)

  Andrei-Aksionov

  authored
  9a2065e

  Copy full SHA for 9a2065e

Commits on Dec 19, 2023

• Apply inference_mode where we kv_cache.reset_parameters (Lightning-AI#837)

  carmocca

  authored
  608e8ed

  Copy full SHA for 608e8ed

• Automated formatting (Lightning-AI#836)

  carmocca

  authored
  602530b

  Copy full SHA for 602530b

Commits on Dec 18, 2023

• Override load_state_dict for the GPT LightningModule (Lightning-AI#806)

  carmocca

  authored
  afca6f5

  Copy full SHA for afca6f5

• Updates to tinyllama pretrain script (Lightning-AI#833)

  awaelchli

  authored
  228c4c3

  Copy full SHA for 228c4c3

• Remove input_ids_no_response from instruction finetuning scripts (Lightning-AI#831)

  awaelchli

  authored
  742f9a7

  Copy full SHA for 742f9a7

• Fix typo in tinyllama tutorial (Lightning-AI#832)

  awaelchli

  authored
  f40b32e

  Copy full SHA for f40b32e

• Script to convert pretrained checkpoints (Lightning-AI#803)

  Show description for 4b0cfd0
  

  awaelchli

  and

  carmocca

  authored
  4b0cfd0

  Copy full SHA for 4b0cfd0

• Precompute the expert mask (Lightning-AI#828)

  carmocca

  authored
  67f3f89

  Copy full SHA for 67f3f89

Commits on Dec 16, 2023

• Update phi tutorial, add a note on input prompt tweaking (Lightning-AI#824)

  Show description for 5ac9322
  

  salykova

  and

  carmocca

  authored
  5ac9322

  Copy full SHA for 5ac9322

• Support multiple epoch iteration in pretrain script (Lightning-AI#820)

  awaelchli

  authored
  b0ccb00

  Copy full SHA for b0ccb00

Commits on Dec 15, 2023

• Create the output dir in merge_lora.py (Lightning-AI#829)

  carmocca

  authored
  cd733bc

  Copy full SHA for cd733bc

• Switch MoE implementation (Lightning-AI#823)

  carmocca

  authored
  74b8df0

  Copy full SHA for 74b8df0

• Use safetensors with Mixtral

  carmocca

  committed
  b23052c

  Copy full SHA for b23052c

• Test fixes for local failures (Lightning-AI#821)

  carmocca

  authored
  0ae5e61

  Copy full SHA for 0ae5e61

• Suggest uninstalling other torch* projects before nightly (Lightning-AI#822)

  carmocca

  authored
  d0f9f8b

  Copy full SHA for d0f9f8b

• Fixes for Mixtral downloading

  carmocca

  committed
  a4a1c9f

  Copy full SHA for a4a1c9f

• Always stop generation on EOS (Lightning-AI#812)

  carmocca

  authored
  d9be087

  Copy full SHA for d9be087

• Modeling fixes for heterogeneous devices (Lightning-AI#814)

  carmocca

  authored
  4d73033

  Copy full SHA for 4d73033

Commits on Dec 14, 2023

• Add Mixtral support (Lightning-AI#816)

  carmocca

  authored
  de51ab1

  Copy full SHA for de51ab1

• Add support for phi-2 model (Lightning-AI#810)

  Show description for 3eb8aae
  

  salykova

  and

  carmocca

  authored
  3eb8aae

  Copy full SHA for 3eb8aae

Pagination

Previous

Next