Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Jan 17, 2025

• fix issue (#435)

  lostkevin

  authored
  e20f015

  Copy full SHA for e20f015

• support virtual PP checkpoint conversion of qwen2-vl (#434)

  lostkevin

  authored
  5d87069

  Copy full SHA for 5d87069

• fix mismatched vision model layers when vpp is set (#432)

  lostkevin

  authored
  77305ea

  Copy full SHA for 77305ea

Commits on Jan 16, 2025

• Upgrade DeepSeek-V2-MoE for facilitating a smooth transition to integrating the DeepSeek-V3-MoE (#431)

  Show description for d981422
  

  jerryli1981

  and

  同润

  authored
  d981422

  Copy full SHA for d981422

Commits on Jan 15, 2025

• add virtual PP, SP, TP comm overlap support for Qwen2-VL (#428)

  Show description for e00b889
  lostkevin

  authored
  e00b889

  Copy full SHA for e00b889

Commits on Jan 9, 2025

• Update moe_layer.py (#402)

  Show description for a40fd6e
  

  cosmosZhou

  and

  jerryli1981

  authored
  a40fd6e

  Copy full SHA for a40fd6e

• [Fix-Issue-397]Qwen2-MoE and DeepSeek-V2 shared experts gradients issue when tp>1 (#425)

  Show description for c12eeef
  

  jerryli1981

  and

  同润

  authored
  c12eeef

  Copy full SHA for c12eeef

• Remove Megatron-LM-240612 (#424)

  Show description for 5ebb944
  

  jerryli1981

  and

  同润

  authored
  5ebb944

  Copy full SHA for 5ebb944

Commits on Jan 8, 2025

• Fix Mistral Convert Issues (#423)

  Show description for 2ac618a
  

  jerryli1981

  and

  同润

  authored
  2ac618a

  Copy full SHA for 2ac618a

• Update qwen2-vl readme and add options in script to support uneven pp conversion (#422)

  Show description for 4318dfa
  

  jerryli1981

  and

  同润

  authored
  4318dfa

  Copy full SHA for 4318dfa

• Fix Mistral Convert Issues (#421)

  Show description for 33855da
  

  jerryli1981

  and

  同润

  authored
  33855da

  Copy full SHA for 33855da

Commits on Jan 6, 2025

• Fix DeepSeek-V2 Finetune Issues (#416)

  Show description for ea7d1e3
  

  jerryli1981

  and

  同润

  authored
  ea7d1e3

  Copy full SHA for ea7d1e3

• update README for attn impl (#415)

  Show description for db5faaf
  lostkevin

  authored
  db5faaf

  Copy full SHA for db5faaf

Commits on Dec 27, 2024

• extend qwen2-vl and bugfix (#407)

  lostkevin

  authored
  d7cab4e

  Copy full SHA for d7cab4e

Commits on Dec 24, 2024

• Fix Mistral Update Issues (#405)

  Show description for 24fa3e9
  

  jerryli1981

  and

  同润

  authored
  24fa3e9

  Copy full SHA for 24fa3e9

• Fix Mistral Update Issues (#404)

  Show description for d0aeb3c
  

  jerryli1981

  and

  同润

  authored
  d0aeb3c

  Copy full SHA for d0aeb3c

Commits on Dec 18, 2024

• Upgrade Mistral MoE and Dense model training (#400)

  Show description for 011e96d
  

  jerryli1981

  and

  同润

  authored
  011e96d

  Copy full SHA for 011e96d

Commits on Dec 5, 2024

• LLama2 70B upgrade and fix nan issue (#391)

  Show description for 9b88cc4
  

  jerryli1981

  and

  同润

  authored
  9b88cc4

  Copy full SHA for 9b88cc4

Commits on Nov 27, 2024

• update readme (#390)

  lostkevin

  authored
  20c4a95

  Copy full SHA for 20c4a95

• Add preview implementation of Qwen2-VL in PAI-Megatron-Patch (#389)

  lostkevin

  authored
  11cf249

  Copy full SHA for 11cf249

Commits on Nov 26, 2024

• bugfix qwen2 grouped gemm error (#388)

  Show description for abe462c
  

  jamindy

  and

  james

  authored
  abe462c

  Copy full SHA for abe462c

Commits on Nov 21, 2024

• Fix LLaVa mcore loss convergence issue (#385)

  Show description for a797b0c
  

  jerryli1981

  and

  同润

  authored
  a797b0c

  Copy full SHA for a797b0c

• Add LLaVa mcore implementation (#384)

  Show description for 51d49f0
  

  jerryli1981

  and

  同润

  authored
  51d49f0

  Copy full SHA for 51d49f0

Commits on Nov 18, 2024

• Fix hang issue due to mismatched attention mask (#379)

  Show description for 3979dbb
  lostkevin

  authored
  3979dbb

  Copy full SHA for 3979dbb

Commits on Nov 14, 2024

• Fix broadcast CPU tensor issue when TP>1 PP=1 in jsonl dataset SFT (#377)

  Show description for 32342a8
  lostkevin

  authored
  32342a8

  Copy full SHA for 32342a8

Commits on Nov 8, 2024

• fix num_seq=None of qwen2 (#376)

  Show description for c77d3b8
  lostkevin

  authored
  c77d3b8

  Copy full SHA for c77d3b8

Commits on Oct 30, 2024

• Add llm auto configurator and apply per seq sft loss for qwen2/2.5 models (#371)

  Show description for bf582d8
  

  jerryli1981

  and

  同润

  authored
  bf582d8

  Copy full SHA for bf582d8

Commits on Oct 24, 2024

• Fix DeepSeekV2Tokenizer bug that padding_side is not set to right (#370)

  Show description for 8003e0c
  

  jerryli1981

  and

  同润

  authored
  8003e0c

  Copy full SHA for 8003e0c

• avoid max(sentence_ids) ValueError: max() arg is an empty sequence (#340)

  Show description for 14ceb0b
  village-way

  authored
  14ceb0b

  Copy full SHA for 14ceb0b

Commits on Oct 23, 2024

• Add --use-cpu-initializatio for Qwen2.5 72B HF2TE ckpt conversion (#367)

  Show description for 4e2da79
  

  jerryli1981

  and

  同润

  authored
  4e2da79

  Copy full SHA for 4e2da79

Commits on Oct 21, 2024

• Fix Qwen2/2.5 te to hf convertion bugs (#365)

  Show description for 8ab426c
  

  jerryli1981

  and

  同润

  authored
  8ab426c

  Copy full SHA for 8ab426c

• add extra clone to fix unexpected large ckpt issue (#364)

  Show description for 672f835
  lostkevin

  authored
  672f835

  Copy full SHA for 672f835

Commits on Oct 16, 2024

• fix convert script to solve empty sample issue (#362)

  lostkevin

  authored
  4b6c687

  Copy full SHA for 4b6c687

Commits on Sep 29, 2024

• Fix Qwen2 tie word embeddings issue (#358)

  Show description for 9d3e557
  

  jerryli1981

  and

  同润

  authored
  9d3e557

  Copy full SHA for 9d3e557

Commits on Sep 26, 2024

• Upgrade deepseek-v2-moe models to support MLA via transformer engine and pipeline ckpts conversion (#355)

  Show description for bcb69cb
  

  jerryli1981

  and

  同润

  authored
  bcb69cb

  Copy full SHA for bcb69cb

Pagination

Previous

Next