Skip to content

Commit

Permalink
updates on the training time
Browse files Browse the repository at this point in the history
  • Loading branch information
oliverwang15 committed Aug 18, 2023
1 parent 520ac8b commit dc7662a
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ We democratize Internet-scale data for financial large language models (FinLLMs)
| FiQA-SA | 0.751 | 0.790 | 0.800 | 0.850 | 0.847 | 0.752 |**0.860** |
| TFNS | - | 0.189 | 0.296 | 0.875 | 0.879 | 0.828 |**0.894** |
| NWGI | - | 0.449 | 0.503 | **0.642** | 0.632 |0.583 |0.636 |
| Devices | 512 × A100 | 64 × A100 | 2048 × A100 | 8 × A100 | 8 × A100 | 8 × A100 |8 × A100 |
| Time | 53 days | 2.5 days | 21 days | 8 hours | 6.47 hours | 4.15 hours |8 hours |
| Devices | 512 × A100 | 64 × A100 | 2048 × A100 | A100 | A100 | A100 |A100 |
| Time | 53 days | 2.5 days | 21 days | 2.36 hours | 6.47 hours | 4.15 hours | 2.47 hours |
| Cost | $2.67 million | $ 14,976 | $4.23 million | $262.4 | $212.2 | $136.12 | $262.4 |
**Cost per GPU hour.** For A100 GPUs, the AWS p4d.24xlarge instance, equipped with 8 A100 GPUs is used as a benchmark to estimate the costs. Note that BloombergGPT also used p4d.24xlarge As of July 11, 2023, the hourly rate for this instance stands at $32.773. Consequently, the estimated cost per GPU hour comes to $32.77 divided by 8, resulting in approximately **$4.10**. With this value as the reference unit price (1 GPU hour). **BloombergGPT estimated cost= 512 x 53 x 24 = 651,264 GPU hours x $4.10 = $2,670,182.40**

Expand Down

0 comments on commit dc7662a

Please sign in to comment.