Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Dec 17, 2021

• Add an argument of dtype to forward_backward functions to specify the dtype used in p2p comm (NVIDIA#1249)

  Show description for b88c507
  crcrpar

  authored
  b88c507

  Copy full SHA for b88c507

Commits on Dec 16, 2021

• version guard (NVIDIA#1253)

  crcrpar

  authored
  e847382

  Copy full SHA for e847382

• Reduce OOM potential and report it if it happens in BERT test (NVIDIA#1250)

  Show description for e0f5ea8
  

  eqy

  and

  crcrpar

  authored
  e0f5ea8

  Copy full SHA for e0f5ea8

Commits on Dec 15, 2021

• Add --threads 4 to extra_compile_args["nvcc"] (NVIDIA#1251)

  Show description for f63dac8
  crcrpar

  authored
  f63dac8

  Copy full SHA for f63dac8

• check model_parallel is initialized in build_model (NVIDIA#1248)

  crcrpar

  authored
  1cd1181

  Copy full SHA for 1cd1181

Commits on Dec 14, 2021

• Faster --fast_multihead_attn build (NVIDIA#1245)

  Show description for 7ec8ed6
  crcrpar

  authored
  7ec8ed6

  Copy full SHA for 7ec8ed6

• check size in kth microbatch (NVIDIA#1247)

  eqy

  authored
  ed94d0b

  Copy full SHA for ed94d0b

Commits on Dec 10, 2021

• Cherry-pick Megatron-LM's changes in pipeline model parallel for T5 (NVIDIA#1232)

  Show description for 0e25fcc
  crcrpar

  authored
  0e25fcc

  Copy full SHA for 0e25fcc

• Minimal gpt pipeline parallel (builds off of minimal_bert_pipeline_parallel) including cpu-offloading (NVIDIA#1222)

  Show description for ab7af05
  

  3 peopleauthored
  ab7af05

  Copy full SHA for ab7af05

Commits on Dec 9, 2021

• Remove THCState from apex/contrib/multihead_attn (NVIDIA#1239)

  Show description for 1203099
  crcrpar

  authored
  1203099

  Copy full SHA for 1203099

• Add fused mixed precision lamb optimizer. (NVIDIA#1237)

  Show description for 3c8f516
  kevinstephano

  authored
  3c8f516

  Copy full SHA for 3c8f516

Commits on Nov 19, 2021

• minimal bert pipeline parallel test (NVIDIA#1216)

  Show description for aa756ce
  eqy

  authored
  aa756ce

  Copy full SHA for aa756ce

• porting GradScaler (NVIDIA#1220)

  Show description for fcae8fa
  

  crcrpar

  and

  erhoo82

  authored
  fcae8fa

  Copy full SHA for fcae8fa

• [POC] Support Megatron-LM's rampup_batch_size argument (NVIDIA#1212)

  Show description for 3533613
  crcrpar

  authored
  3533613

  Copy full SHA for 3533613

Commits on Nov 10, 2021

• conditionally import amp_C (NVIDIA#1211)

  crcrpar

  authored
  25bfcb9

  Copy full SHA for 25bfcb9

• check in (NVIDIA#1210)

  eqy

  authored
  2205cff

  Copy full SHA for 2205cff

• check in (NVIDIA#1205)

  eqy

  authored
  fa8bd7e

  Copy full SHA for fa8bd7e

Commits on Oct 27, 2021

• FastLayerNorm compat with autocast (NVIDIA#1203)

  Show description for ae75763
  

  crcrpar

  and

  yjk21

  authored
  ae75763

  Copy full SHA for ae75763

• Pipeline Model Parallel (NVIDIA#1202)

  Show description for 63d5dd6
  

  3 peopleauthored
  63d5dd6

  Copy full SHA for 63d5dd6

Commits on Oct 23, 2021

• Use out-of-place to avoid D2D copy in tensor parallel cross entropy (NVIDIA#1198)

  Show description for 3303b3e
  

  crcrpar

  and

  eqy

  authored
  3303b3e

  Copy full SHA for 3303b3e

Commits on Oct 18, 2021

• remove THC headers/functions (NVIDIA#1192)

  Show description for 0c7d8e3
  crcrpar

  authored
  0c7d8e3

  Copy full SHA for 0c7d8e3

Commits on Oct 16, 2021

• replace (NVIDIA#1191)

  crcrpar

  authored
  60821f5

  Copy full SHA for 60821f5

Commits on Oct 14, 2021

• change chunking scheme for full-allreduce case, add parameter order argument, both to enable contiguous chunking of allgather (NVIDIA#1190)

  seryilmaz

  authored
  1d5f7e5

  Copy full SHA for 1d5f7e5

• Fix dist lamb (NVIDIA#1185)

  Show description for d9a46fd
  nanz-nv

  authored
  d9a46fd

  Copy full SHA for d9a46fd

Commits on Oct 13, 2021

• check in (NVIDIA#1189)

  eqy

  authored
  4e9fae9

  Copy full SHA for 4e9fae9

Commits on Oct 8, 2021

• Remove custom_fwd/custom_bwd from fused softmax (NVIDIA#1188)

  Show description for 14ccf59
  crcrpar

  authored
  14ccf59

  Copy full SHA for 14ccf59

• check in (NVIDIA#1187)

  eqy

  authored
  3ad9db2

  Copy full SHA for 3ad9db2

Commits on Oct 7, 2021

• Update layer_norm_cuda_kernel.cu (NVIDIA#1184)

  eqy

  authored
  5adf7bc

  Copy full SHA for 5adf7bc

Commits on Oct 6, 2021

• ColumnParallelLinearWithAsyncAllreduce autocast support (NVIDIA#1183)

  Show description for b3da603
  crcrpar

  authored
  b3da603

  Copy full SHA for b3da603

Commits on Oct 2, 2021

• transformer utils (NVIDIA#1181)

  Show description for 365fdc1
  

  5 peopleauthored
  365fdc1

  Copy full SHA for 365fdc1

Commits on Sep 30, 2021

• use cuda caching allocator from pytorch (NVIDIA#1180)

  xwang233

  authored
  bdac244

  Copy full SHA for bdac244

Commits on Sep 28, 2021

• cleanup missing THCDeviceUtils.cuh header (NVIDIA#1177)

  xwang233

  authored
  2a559c5

  Copy full SHA for 2a559c5

Commits on Sep 24, 2021

• Fix typo in contrib FusedLamb. (NVIDIA#1172)

  romerojosh

  authored
  70d4a0b

  Copy full SHA for 70d4a0b

• THCDeviceUtils.cuh -> ATen/cuda/DeviceUtils.cuh (NVIDIA#1173)

  crcrpar

  authored
  76daa45

  Copy full SHA for 76daa45

Commits on Sep 8, 2021

• enable ninja (NVIDIA#1164)

  Show description for 9ce0a10
  crcrpar

  authored
  9ce0a10

  Copy full SHA for 9ce0a10

Pagination

Previous

Next