Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Mar 23, 2023

• Overlap first param sync with optimizer step in distributed Adam (NVIDIA#1621)

  Show description for 7150e20
  timmoon10

  authored
  7150e20

  Copy full SHA for 7150e20

• check for empty params in group (NVIDIA#1596)

  Show description for 0420735
  

  3 peopleauthored
  0420735

  Copy full SHA for 0420735

Commits on Mar 22, 2023

• Revert "Add support for FP32 master weights in FusedAdam optimizer (NVIDIA#1623)" (NVIDIA#1624)

  Show description for 4060aef
  crcrpar

  authored
  4060aef

  Copy full SHA for 4060aef

• Add support for FP32 master weights in FusedAdam optimizer (NVIDIA#1623)

  Show description for 6952004
  

  4 peopleauthored
  6952004

  Copy full SHA for 6952004

Commits on Mar 20, 2023

• p2p communication overlap at interleaved pipelining (NVIDIA#1616)

  Show description for d8643ef
  erhoo82

  authored
  d8643ef

  Copy full SHA for d8643ef

Commits on Mar 17, 2023

• Align distopt communication in interleaved pipeline parallelism (NVIDIA#1611)

  Show description for 27de66c
  

  timmoon10

  and

  crcrpar

  authored
  27de66c

  Copy full SHA for 27de66c

• [fmha] Use 8bit comparison to generate dropout mask (NVIDIA#1614)

  vasunvidia

  authored
  07b2935

  Copy full SHA for 07b2935

Commits on Mar 13, 2023

• ConvFrozenScaleBiasReLU fusion (NVIDIA#1608)

  Show description for 623315a
  

  minitu

  and

  Jaemin Choi

  authored
  623315a

  Copy full SHA for 623315a

Commits on Mar 7, 2023

• [FMHA] Remove zero fill for softmax output (NVIDIA#1565)

  Show description for b1c7600
  vasunvidia

  authored
  b1c7600

  Copy full SHA for b1c7600

• s/maximimize/maximize/ (NVIDIA#1604)

  Show description for be819ea
  EIFY

  authored
  be819ea

  Copy full SHA for be819ea

Commits on Mar 6, 2023

• gracefully skipping permutations when torch.fx fails tracing (NVIDIA#1595)

  jpool-nv

  authored
  e29fb45

  Copy full SHA for e29fb45

Commits on Feb 28, 2023

• Update layers.py to ensure grad_output is contiguous (NVIDIA#1601)

  Show description for 95cea8e
  markelsanz14

  authored
  95cea8e

  Copy full SHA for 95cea8e

Commits on Feb 25, 2023

• use a single communicator for amax reduction in both TP and DP domains (NVIDIA#1597)

  erhoo82

  authored
  eec7250

  Copy full SHA for eec7250

• Decouple optimizer state and grad dtypes in distributed Adam optimizer (NVIDIA#1575)

  Show description for 03c9d80
  timmoon10

  authored
  03c9d80

  Copy full SHA for 03c9d80

Commits on Feb 16, 2023

• Use a separate communicator for AMAX reduction (NVIDIA#1585)

  Show description for 0c8400a
  erhoo82

  authored
  0c8400a

  Copy full SHA for 0c8400a

• use torch.tensor to create a tensor with initializer values (NVIDIA#1588)

  Show description for 6943fd2
  crcrpar

  authored
  6943fd2

  Copy full SHA for 6943fd2

Commits on Feb 15, 2023

• torch.empty() must have args (NVIDIA#1584)

  NouamaneTazi

  authored
  93bb36a

  Copy full SHA for 93bb36a

Commits on Feb 13, 2023

• replace torch.Tensor with torch.empty (NVIDIA#1578)

  Show description for ba027dd
  NouamaneTazi

  authored
  ba027dd

  Copy full SHA for ba027dd

Commits on Feb 9, 2023

• decrease warnings & remove append_nvcc_threads from setup.py (NVIDIA#1581)

  Show description for 0145d69
  crcrpar

  authored
  0145d69

  Copy full SHA for 0145d69

Commits on Feb 7, 2023

• return none-list input as as: pass the data loader iterator to training function (NVIDIA#1580)

  Show description for a32d7a6
  erhoo82

  authored
  a32d7a6

  Copy full SHA for a32d7a6

• Distributed optimizer overlaps forward compute and param all-gather (NVIDIA#1559)

  Show description for c0a0b0f
  timmoon10

  authored
  c0a0b0f

  Copy full SHA for c0a0b0f

Commits on Feb 3, 2023

• [fused softmax] integer overflow and non-contiguous memory (NVIDIA#1568)

  Show description for 5c9625c
  cyanguwa

  authored
  5c9625c

  Copy full SHA for 5c9625c

Commits on Jan 31, 2023

• Capturable Adam optimizer (NVIDIA#1560)

  Show description for ad68a2f
  gilshm

  authored
  ad68a2f

  Copy full SHA for ad68a2f

Commits on Jan 26, 2023

• update test routines (NVIDIA#1567)

  Aidyn-A

  authored
  537424d

  Copy full SHA for 537424d

• bump cutlass to v2.11.0 for contrib/multihead_attn (NVIDIA#1570)

  Show description for 536e549
  crcrpar

  authored
  536e549

  Copy full SHA for 536e549

Commits on Jan 20, 2023

• Optimizing distributed Adam when running with one work queue (NVIDIA#1551)

  Show description for 75f401e
  timmoon10

  authored
  75f401e

  Copy full SHA for 75f401e

Commits on Jan 19, 2023

• Extend softmax fusion seq length to 16384 (NVIDIA#1558)

  Show description for e1e0531
  yaoyu-33

  authored
  e1e0531

  Copy full SHA for e1e0531

Commits on Dec 9, 2022

• change param of list of tensor to tensor for _make_nccl_premul_sum (NVIDIA#1542)

  Show description for 91fcaaf
  crcrpar

  authored
  91fcaaf

  Copy full SHA for 91fcaaf

Commits on Dec 8, 2022

• Refactor run_transformer bert minimal and gpt minimal tests (NVIDIA#1540)

  Show description for 85af77c
  Fuzzkatt

  authored
  85af77c

  Copy full SHA for 85af77c

Commits on Nov 30, 2022

• Merge pull request NVIDIA#1543 from timmoon10/peer-halo-exchange-perf-optimization

  Show description for 190a446
  thorjohnsen

  authored
  190a446

  Copy full SHA for 190a446

Commits on Nov 22, 2022

• Update megatron fused softmax follow megatron-lm (NVIDIA#1539)

  Show description for abeca58
  yaoyu-33

  authored
  abeca58

  Copy full SHA for abeca58

Commits on Nov 21, 2022

• Avoid index bookkeeping in halo exchange kernel if data is contiguous

  timmoon10

  committed
  fa34475

  Copy full SHA for fa34475

• [contrib] fix reduce_scatter_tensor fallouts in DistributedFusedLamb (NVIDIA#1536)

  Show description for c216175
  

  crcrpar

  and

  mikolajblaz

  authored
  c216175

  Copy full SHA for c216175

Commits on Nov 18, 2022

• Optimize peer memory halo exchange kernel

  Show description for 0bd620e
  timmoon10

  committed
  0bd620e

  Copy full SHA for 0bd620e

Commits on Nov 17, 2022

• add setter for virtual world size (NVIDIA#1541)

  Show description for 3229126
  

  timmoon10

  and

  ericharper

  authored
  3229126

  Copy full SHA for 3229126

Pagination

Previous

Next