Model save

README.md

@@ -1,11 +1,10 @@
 ---
 base_model: meta-llama/Meta-Llama-3.1-8B
 datasets:
-- llama-duo/synth_classification_dataset_dedup
+- generator
 library_name: peft
 license: llama3.1
 tags:
-- alignment-handbook
 - trl
 - sft
 - generated_from_trainer
@@ -19,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # llama3.1-8b-classification-gpt4o-100k
 
-This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on the llama-duo/synth_classification_dataset_dedup dataset.
+This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.8520
+- Loss: 3.0330
 
 ## Model description
 
@@ -56,18 +55,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.4961        | 0.9978 | 225  | 1.7708          |
-| 1.3952        | 2.0    | 451  | 1.7770          |
-| 1.3491        | 2.9978 | 676  | 1.7484          |
-| 1.3025        | 4.0    | 902  | 1.7902          |
-| 1.2904        | 4.9978 | 1127 | 1.7997          |
-| 1.2729        | 6.0    | 1353 | 1.8170          |
-| 1.2451        | 6.9978 | 1578 | 1.8180          |
-| 1.229         | 8.0    | 1804 | 1.8372          |
-| 1.2239        | 8.9978 | 2029 | 1.8482          |
-| 1.2051        | 9.9778 | 2250 | 1.8520          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.2062        | 1.0   | 296  | 1.6781          |
+| 1.1339        | 2.0   | 592  | 1.6897          |
+| 1.0779        | 3.0   | 888  | 1.7536          |
+| 1.0043        | 4.0   | 1184 | 1.8225          |
+| 0.9288        | 5.0   | 1480 | 2.0044          |
+| 0.8437        | 6.0   | 1776 | 2.1710          |
+| 0.7654        | 7.0   | 2072 | 2.4080          |
+| 0.7117        | 8.0   | 2368 | 2.6554          |
+| 0.6916        | 9.0   | 2664 | 2.9172          |
+| 0.6652        | 10.0  | 2960 | 3.0330          |
 
 
 ### Framework versions

all_results.json

@@ -1,14 +1,9 @@
 {
-    "epoch": 9.977827050997783,
-    "eval_loss": 1.8520119190216064,
-    "eval_runtime": 0.3553,
-    "eval_samples": 16,
-    "eval_samples_per_second": 2.814,
-    "eval_steps_per_second": 2.814,
-    "total_flos": 3.3259687719144e+18,
-    "train_loss": 1.3362829395929972,
-    "train_runtime": 6815.0283,
+    "epoch": 10.0,
+    "total_flos": 4.416382035459834e+18,
+    "train_loss": 0.922980490487975,
+    "train_runtime": 12382.7598,
     "train_samples": 92634,
-    "train_samples_per_second": 10.572,
-    "train_steps_per_second": 0.33
+    "train_samples_per_second": 7.645,
+    "train_steps_per_second": 0.239
 }

train_results.json

@@ -1,9 +1,9 @@
 {
-    "epoch": 9.977827050997783,
-    "total_flos": 3.3259687719144e+18,
-    "train_loss": 1.3362829395929972,
-    "train_runtime": 6815.0283,
+    "epoch": 10.0,
+    "total_flos": 4.416382035459834e+18,
+    "train_loss": 0.922980490487975,
+    "train_runtime": 12382.7598,
     "train_samples": 92634,
-    "train_samples_per_second": 10.572,
-    "train_steps_per_second": 0.33
+    "train_samples_per_second": 7.645,
+    "train_steps_per_second": 0.239
 }