Skip to content

Commit

Permalink
change grad accum to default off because i think it just confuses eve…
Browse files Browse the repository at this point in the history
…ryone
karpathy committed Feb 2, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent d01863e commit d8b1a94
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion train.py
Original file line number Diff line number Diff line change
@@ -45,7 +45,7 @@
wandb_run_name = 'gpt2' # 'run' + str(time.time())
# data
dataset = 'openwebtext'
gradient_accumulation_steps = 5 # used to simulate larger batch sizes
gradient_accumulation_steps = 1 # used to simulate larger batch sizes
batch_size = 12 # if gradient_accumulation_steps > 1, this is the micro-batch size
block_size = 1024
# model

0 comments on commit d8b1a94

Please sign in to comment.