Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Feb 13, 2023

• replace torch.Tensor with torch.empty (NVIDIA#1578)

  Show description for ba027dd
  NouamaneTazi

  authored
  ba027dd

  Copy full SHA for ba027dd

Commits on Feb 9, 2023

• decrease warnings & remove append_nvcc_threads from setup.py (NVIDIA#1581)

  Show description for 0145d69
  crcrpar

  authored
  0145d69

  Copy full SHA for 0145d69

Commits on Feb 7, 2023

• return none-list input as as: pass the data loader iterator to training function (NVIDIA#1580)

  Show description for a32d7a6
  erhoo82

  authored
  a32d7a6

  Copy full SHA for a32d7a6

• Distributed optimizer overlaps forward compute and param all-gather (NVIDIA#1559)

  Show description for c0a0b0f
  timmoon10

  authored
  c0a0b0f

  Copy full SHA for c0a0b0f

Commits on Feb 3, 2023

• [fused softmax] integer overflow and non-contiguous memory (NVIDIA#1568)

  Show description for 5c9625c
  cyanguwa

  authored
  5c9625c

  Copy full SHA for 5c9625c

Commits on Jan 31, 2023

• Capturable Adam optimizer (NVIDIA#1560)

  Show description for ad68a2f
  gilshm

  authored
  ad68a2f

  Copy full SHA for ad68a2f

Commits on Jan 26, 2023

• update test routines (NVIDIA#1567)

  Aidyn-A

  authored
  537424d

  Copy full SHA for 537424d

• bump cutlass to v2.11.0 for contrib/multihead_attn (NVIDIA#1570)

  Show description for 536e549
  crcrpar

  authored
  536e549

  Copy full SHA for 536e549

Commits on Jan 20, 2023

• Optimizing distributed Adam when running with one work queue (NVIDIA#1551)

  Show description for 75f401e
  timmoon10

  authored
  75f401e

  Copy full SHA for 75f401e

Commits on Jan 19, 2023

• Extend softmax fusion seq length to 16384 (NVIDIA#1558)

  Show description for e1e0531
  yaoyu-33

  authored
  e1e0531

  Copy full SHA for e1e0531

Commits on Dec 9, 2022

• change param of list of tensor to tensor for _make_nccl_premul_sum (NVIDIA#1542)

  Show description for 91fcaaf
  crcrpar

  authored
  91fcaaf

  Copy full SHA for 91fcaaf

Commits on Dec 8, 2022

• Refactor run_transformer bert minimal and gpt minimal tests (NVIDIA#1540)

  Show description for 85af77c
  Fuzzkatt

  authored
  85af77c

  Copy full SHA for 85af77c

Commits on Nov 30, 2022

• Merge pull request NVIDIA#1543 from timmoon10/peer-halo-exchange-perf-optimization

  Show description for 190a446
  thorjohnsen

  authored
  190a446

  Copy full SHA for 190a446

Commits on Nov 22, 2022

• Update megatron fused softmax follow megatron-lm (NVIDIA#1539)

  Show description for abeca58
  yaoyu-33

  authored
  abeca58

  Copy full SHA for abeca58

Commits on Nov 21, 2022

• Avoid index bookkeeping in halo exchange kernel if data is contiguous

  timmoon10

  committed
  fa34475

  Copy full SHA for fa34475

• [contrib] fix reduce_scatter_tensor fallouts in DistributedFusedLamb (NVIDIA#1536)

  Show description for c216175
  

  crcrpar

  and

  mikolajblaz

  authored
  c216175

  Copy full SHA for c216175

Commits on Nov 18, 2022

• Optimize peer memory halo exchange kernel

  Show description for 0bd620e
  timmoon10

  committed
  0bd620e

  Copy full SHA for 0bd620e

Commits on Nov 17, 2022

• add setter for virtual world size (NVIDIA#1541)

  Show description for 3229126
  

  timmoon10

  and

  ericharper

  authored
  3229126

  Copy full SHA for 3229126

• Perform dist Adam communication in side stream (NVIDIA#1528)

  Show description for d79f69a
  timmoon10

  authored
  d79f69a

  Copy full SHA for d79f69a

Commits on Nov 14, 2022

• [README] Add a list of install options (NVIDIA#1492)

  Show description for a9950b1
  crcrpar

  authored
  a9950b1

  Copy full SHA for a9950b1

Commits on Oct 27, 2022

• update exception message (NVIDIA#1524)

  Show description for 6a40a0a
  crcrpar

  authored
  6a40a0a

  Copy full SHA for 6a40a0a

• check is_ucc_available (NVIDIA#1523)

  Show description for e76ade0
  crcrpar

  authored
  e76ade0

  Copy full SHA for e76ade0

• Remove weight gradients if model is frozen for linear layer (NVIDIA#1517)

  Show description for 36ffac0
  

  3 peopleauthored
  36ffac0

  Copy full SHA for 36ffac0

• update run_transformer tests to default to using pytorch native UCC instead of torch_ucc (NVIDIA#1495)

  Show description for 0d06a73
  Fuzzkatt

  authored
  0d06a73

  Copy full SHA for 0d06a73

• Support overlapped grad sync with Megatron interleaved pipeline parallelism (NVIDIA#1514)

  Show description for f683961
  timmoon10

  authored
  f683961

  Copy full SHA for f683961

Commits on Oct 26, 2022

• Use all_gather_into_tensor and reduce_scatter_tensor (NVIDIA#1513)

  Show description for f680fab
  Aidyn-A

  authored
  f680fab

  Copy full SHA for f680fab

Commits on Oct 25, 2022

• Partial activation checkpoint (NVIDIA#1511)

  Show description for ee42e2d
  

  crcrpar

  and

  erhoo82

  authored
  ee42e2d

  Copy full SHA for ee42e2d

• Fix bug with grad clipping and distributed Adam (NVIDIA#1512)

  Show description for da9f5ae
  timmoon10

  authored
  da9f5ae

  Copy full SHA for da9f5ae

Commits on Oct 13, 2022

• add tearDown (NVIDIA#1508)

  Aidyn-A

  authored
  2c3b8f9

  Copy full SHA for 2c3b8f9

Commits on Oct 12, 2022

• [transformer] Allow for skipping stream sync (NVIDIA#1505)

  Show description for 806f9b0
  

  3 peopleauthored
  806f9b0

  Copy full SHA for 806f9b0

Commits on Oct 10, 2022

• Deprecate outdated features (NVIDIA#1506)

  Show description for 48bf0df
  crcrpar

  authored
  48bf0df

  Copy full SHA for 48bf0df

Commits on Oct 6, 2022

• check CUDA_HOME before using it (NVIDIA#1499)

  Show description for b213772
  crcrpar

  authored
  b213772

  Copy full SHA for b213772

• Update for CUDA 11.8 (NVIDIA#1502)

  Show description for 5f1a73e
  

  3 peopleauthored
  5f1a73e

  Copy full SHA for 5f1a73e

Commits on Oct 2, 2022

• Replace usage of internal with public APIs (NVIDIA#1496)

  Show description for 6816ef6
  kumpera

  authored
  6816ef6

  Copy full SHA for 6816ef6

Commits on Sep 22, 2022

• update fused_dense to use torch.autocast instead of apex amp (NVIDIA#1490)

  Show description for 1d77111
  Fuzzkatt

  authored
  1d77111

  Copy full SHA for 1d77111

Pagination

Previous

Next