Commits

Branch selector

 master

User selector

karpathy

Datepicker

All time

Commit History

Commits on Jan 25, 2023

• allow the prompt to compe from a file

  karpathy

  committed
  e0c689c

  Copy full SHA for e0c689c

• allow sample.py to init from a pretrained gpt2 checkpoints as well, in similar style to train.py

  karpathy

  committed
  21675d7

  Copy full SHA for 21675d7

Commits on Jan 22, 2023

• Merge pull request karpathy#82 from danielgross/master

  Show description for 6c40a08
  karpathy

  authored
  6c40a08

  Copy full SHA for 6c40a08

Commits on Jan 20, 2023

• Merge pull request karpathy#71 from cchan/patch-1

  Show description for 3611338
  karpathy

  authored
  3611338

  Copy full SHA for 3611338

• make mentions of mps in docs. ty good people in issue karpathy#28

  karpathy

  committed
  1f77d03

  Copy full SHA for 1f77d03

• Merge pull request karpathy#73 from danielgross/master

  Show description for a6bffee
  karpathy

  authored
  a6bffee

  Copy full SHA for a6bffee

Commits on Jan 19, 2023

• for consistency with previous commit

  karpathy

  committed
  2c7806d

  Copy full SHA for 2c7806d

• Merge pull request karpathy#57 from ryouze/patch-1

  Show description for c1c20a0
  karpathy

  authored
  c1c20a0

  Copy full SHA for c1c20a0

• Merge pull request karpathy#66 from PWhiddy/patch-1

  Show description for 9e150b8
  karpathy

  authored
  9e150b8

  Copy full SHA for 9e150b8

Commits on Jan 18, 2023

• fix temperature comment, slightly wrong

  karpathy

  committed
  8dd2061

  Copy full SHA for 8dd2061

• the badge is a bit ugly, move it down to troubleshooting section

  karpathy

  committed
  2b083fb

  Copy full SHA for 2b083fb

• screwed up the link, fix

  karpathy

  committed
  aa8e4c2

  Copy full SHA for aa8e4c2

• experimenting with badges, and discord link to start specifically. issues sometimes feel a little too heavy

  karpathy

  committed
  6dab32c

  Copy full SHA for 6dab32c

Commits on Jan 16, 2023

• add docs on multinode training to main README too

  karpathy

  committed
  7f74652

  Copy full SHA for 7f74652

• small tweaks to docs and variable names stylistically

  karpathy

  committed
  46ce997

  Copy full SHA for 46ce997

• clarify that these should be run on two separate machines

  karpathy

  committed
  684800d

  Copy full SHA for 684800d

• docs for multinode ddp

  karpathy

  committed
  9352df2

  Copy full SHA for 9352df2

• get rid of gpu_id, the world is more complicated than that when world_size > 8

  karpathy

  committed
  c3dddbf

  Copy full SHA for c3dddbf

• local rank -> rank

  karpathy

  committed
  f5e6ac8

  Copy full SHA for f5e6ac8

Commits on Jan 15, 2023

• add gradient accumulation support to simulate larger batch sizes. ty @VHellendoorn for original PR

  karpathy

  committed
  cf99914

  Copy full SHA for cf99914

Commits on Jan 14, 2023

• add note of caution for the produced warning, investigate later

  karpathy

  committed
  89da79e

  Copy full SHA for 89da79e

• a bit better settings... for a single gpu at least. these settings would fry a simple cpu though i think

  karpathy

  committed
  7d7ded2

  Copy full SHA for 7d7ded2

• fix bug... if topk > vocab_size, torch.topk will throw error

  karpathy

  committed
  91d0251

  Copy full SHA for 91d0251

• correctly propagate the vocab_size from the rendered dataset into the model args

  karpathy

  committed
  57735f5

  Copy full SHA for 57735f5

• reverse the order, making sure that the final layer init is preserved, and becomes the token embedding instead of the other way around. otherwise the loss can be all messed up from a bad init

  karpathy

  committed
  43b37fd

  Copy full SHA for 43b37fd

• tie the weights of lm_head.weight and transformer.wte.weight, i.e. the last linear layer of decoder and the token embeddings.

  karpathy

  committed
  7c82885

  Copy full SHA for 7c82885

Commits on Jan 13, 2023

• it's true

  karpathy

  committed
  32b4f08

  Copy full SHA for 32b4f08

• more scaling laws, clarification, and add simple interpolation of Approach 2

  karpathy

  committed
  3e0fd42

  Copy full SHA for 3e0fd42

Commits on Jan 12, 2023

• inference time mini-optimization low-hanging fruit ty @jxtps for raising: when we are running inference we can apply lm_head on only the very last token

  karpathy

  committed
  8f85b83

  Copy full SHA for 8f85b83

• meant to set always_save_checkpoint to False instead, so we only write when val improves

  karpathy

  committed
  e21cbf8

  Copy full SHA for e21cbf8

• including transformers as a dependency of the repo as well

  karpathy

  committed
  c1ac2d5

  Copy full SHA for c1ac2d5

• add note about windows and pytorch 2.0 and torch compile in general

  karpathy

  committed
  7f51d17

  Copy full SHA for 7f51d17

Commits on Jan 11, 2023

• oh no nanoGPT is trending quickly explain the character-level functionality I added late last night

  karpathy

  committed
  bb49751

  Copy full SHA for bb49751

• add support for character-level language models, a new character-level shakespeare dataset, a new config file that shows how to train a character-level baby GPT on it, and adjust the sample functio…

  Show description for d17350a
  karpathy

  committed
  d17350a

  Copy full SHA for d17350a

• guess the config from globals() and log all of it with wandb

  karpathy

  committed
  c2a402f

  Copy full SHA for c2a402f

Pagination

Previous

Next