Skip to content

Commit

Permalink
change "Content Length" to "Context Length MODEL_CARD.md
Browse files Browse the repository at this point in the history
  • Loading branch information
yonashub authored Oct 15, 2023
1 parent 556949f commit f9ddb1d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion MODEL_CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Meta developed and released the Llama 2 family of large language models (LLMs),

**Model Architecture** Llama 2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

||Training Data|Params|Content Length|GQA|Tokens|LR|
||Training Data|Params|Context Length|GQA|Tokens|LR|
|---|---|---|---|---|---|---|
Llama 2|*A new mix of publicly available online data*|7B|4k|&#10007;|2.0T|3.0 x 10<sup>-4</sup>
Llama 2|*A new mix of publicly available online data*|13B|4k|&#10007;|2.0T|3.0 x 10<sup>-4</sup>
Expand Down

0 comments on commit f9ddb1d

Please sign in to comment.