Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Jan 13, 2023

• more scaling laws, clarification, and add simple interpolation of Approach 2

  karpathy

  committed
  3e0fd42

  Copy full SHA for 3e0fd42

Commits on Jan 12, 2023

• inference time mini-optimization low-hanging fruit ty @jxtps for raising: when we are running inference we can apply lm_head on only the very last token

  karpathy

  committed
  8f85b83

  Copy full SHA for 8f85b83

• meant to set always_save_checkpoint to False instead, so we only write when val improves

  karpathy

  committed
  e21cbf8

  Copy full SHA for e21cbf8

• including transformers as a dependency of the repo as well

  karpathy

  committed
  c1ac2d5

  Copy full SHA for c1ac2d5

• add note about windows and pytorch 2.0 and torch compile in general

  karpathy

  committed
  7f51d17

  Copy full SHA for 7f51d17

Commits on Jan 11, 2023

• oh no nanoGPT is trending quickly explain the character-level functionality I added late last night

  karpathy

  committed
  bb49751

  Copy full SHA for bb49751

• add support for character-level language models, a new character-level shakespeare dataset, a new config file that shows how to train a character-level baby GPT on it, and adjust the sample functio…

  Show description for d17350a
  karpathy

  committed
  d17350a

  Copy full SHA for d17350a

• guess the config from globals() and log all of it with wandb

  karpathy

  committed
  c2a402f

  Copy full SHA for c2a402f

Commits on Jan 8, 2023

• adjust the readme to reflect changes in the autocast branch

  karpathy

  committed
  8b2e622

  Copy full SHA for 8b2e622

• copy pasting what seems to work to bench,sample as well. ty @lantiga

  karpathy

  committed
  b77c2e8

  Copy full SHA for b77c2e8

• add device and dtype support to train.py args

  karpathy

  committed
  a855d31

  Copy full SHA for a855d31

• Merge pull request karpathy#20 from lantiga/wandb-optional-import

  Show description for e7cd674
  karpathy

  authored
  e7cd674

  Copy full SHA for e7cd674

• Move conditional import

  lantiga

  committed
  09f1f45

  Copy full SHA for 09f1f45

• Make wandb import conditioned to wandb_log=True

  lantiga

  committed
  aba47f0

  Copy full SHA for aba47f0

• ran readme through spellchecker heh

  karpathy

  committed
  e53b9d2

  Copy full SHA for e53b9d2

• tune the readme with new header image and the loss curve for 124M

  karpathy

  committed
  df3b8a5

  Copy full SHA for df3b8a5

Commits on Jan 7, 2023

• progress! based on chinchilla author correspondence

  karpathy

  committed
  d56bdf0

  Copy full SHA for d56bdf0

Commits on Jan 6, 2023

• small tweaks to notebook

  karpathy

  committed
  27fc6a4

  Copy full SHA for 27fc6a4

• update scaling laws. basically i can't reproduce any of params, flops, or scaling laws of the Chinchilla paper atm...

  karpathy

  committed
  69d1a5f

  Copy full SHA for 69d1a5f

Commits on Jan 5, 2023

• minor args re-arranging and removing some spurious ones like wandb entity ty @tcapelle

  karpathy

  committed
  9629093

  Copy full SHA for 9629093

• Merge pull request karpathy#19 from nat/patch-1

  Show description for 529c967
  karpathy

  authored
  529c967

  Copy full SHA for 529c967

• shuttling the poor mans configurator aside into its own file and adding it to all of train,sample,bench. because i am leaving args in globals() so i can avoid having to prepend every single variabl…

  Show description for d562b3e
  karpathy

  committed
  d562b3e

  Copy full SHA for d562b3e

• Strip unwanted prefix from state keys when loading model

  nat

  committed
  2b9e168

  Copy full SHA for 2b9e168

Commits on Jan 4, 2023

• mention current 8GPU SOTA and shuffle sections a bit

  karpathy

  committed
  ab04701

  Copy full SHA for ab04701

• Merge pull request karpathy#16 from jorahn/patch-1

  Show description for 1eefbb2
  karpathy

  authored
  1eefbb2

  Copy full SHA for 1eefbb2

• Update README.md

  jorahn

  authored
  26aa5f3

  Copy full SHA for 26aa5f3

• add a notebook trying to reproduce chinchilla scaling laws. I can't get the numbers to be exactly right, have to look at more

  karpathy

  committed
  c72ecf5

  Copy full SHA for c72ecf5

Commits on Jan 3, 2023

• ty lambda labs

  karpathy

  committed
  5acba4b

  Copy full SHA for 5acba4b

• adding few more dependencies

  karpathy

  committed
  97fc426

  Copy full SHA for 97fc426

• better hyperparams for gpt2 124M model on A100 40GB. still uncertain about max_iters especially, and a bit about weight decay, betas

  karpathy

  committed
  9f95aca

  Copy full SHA for 9f95aca

• flesh out the remaining TODOs in readme a bit more

  karpathy

  committed
  b45eec3

  Copy full SHA for b45eec3

Commits on Jan 2, 2023

• disabling torch.jit.script here for massive performance boost when using torch.compile, our default. see issue karpathy#11. thanks @vgoklani for flagging

  karpathy

  committed
  177d5f7

  Copy full SHA for 177d5f7

• reshuffle args inside sample.py

  karpathy

  committed
  ea4de19

  Copy full SHA for ea4de19

• add a patch to fix mysterious unwanted prefix in state dict? maybe remove later

  karpathy

  committed
  ec9b1f8

  Copy full SHA for ec9b1f8

• rename compile_model to compile, shroter, version 2 stragglers

  karpathy

  committed
  41184a2

  Copy full SHA for 41184a2

Pagination

Previous

Next