togethercomputer
/

RedPajama-INCITE-7B-Base

Text Generation

text-generation-inference

Model card Files Files and versions

juewang commited on Jun 6, 2023

Commit

f5e6c75

·

1 Parent(s): a5605bd

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -184,7 +184,7 @@ Please refer to [togethercomputer/RedPajama-Data-1T](https://huggingface.co/data
 - **Optimizer:** Apex FusedAdam
 - **Parallelism:** Pipeline parallel 12, tensor parallel 2
 - **Gradient Accumulations**: 8 (global batch size 4M tokens)
-- **Num of Tokens:** 1T Tokens
 - **Learning rate:** 0.00012
 ## Benchmark

 - **Optimizer:** Apex FusedAdam
 - **Parallelism:** Pipeline parallel 12, tensor parallel 2
 - **Gradient Accumulations**: 8 (global batch size 4M tokens)
+- **Num of Tokens:** 1.001T Tokens
 - **Learning rate:** 0.00012
 ## Benchmark