End of training

README.md

@@ -28,14 +28,13 @@ strict: false
 datasets:
   - path: AI-MO/NuminaMath-CoT
     type: sharegpt.load_ultrachat
+    conversation: llama3
 
-chat_template: llama3
 dataset_prepared_path: /scratch/bf996/axolotl/datasets/numina
-val_set_size: 0.001
 output_dir: /scratch/bf996/axolotl/outputs/numina
+
 sequence_len: 8192
 sample_packing: true
-eval_sample_packing: false
 pad_to_sequence_len: true
 
 wandb_project: lm-evals
@@ -87,12 +86,10 @@ special_tokens:
 
 </details><br>
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/nyu-dice-lab/lm-evals/runs/ghe48g78)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/nyu-dice-lab/lm-evals/runs/sfrh2mks)
 # Llama-3-8B-NuminaCoT
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.3943
 
 ## Model description
 
@@ -123,14 +120,10 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 100
-- training_steps: 3460
+- training_steps: 3350
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 0.4379        | 1.0130 | 1826 | 0.3994          |
-| 0.3928        | 1.9064 | 3460 | 0.3943          |
 
 
 ### Framework versions

pytorch_model-00001-of-00004.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5e0cc9f6527c9af1389c4c041f4740ff6b6bed5353b7f21e2015cbec77ed78d4
+oid sha256:caa6ed1336169130f4db3b1b819ff8b1bbd537a7642b83aba8de01256075df11
 size 4976718466

pytorch_model-00002-of-00004.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91c6851988a44e6d4a31111486ddbe18a14e4f376981b7650c49791ecf838515
+oid sha256:6b46d2f759bd11cc9bf51d779db1e3146b39bf78bb98c5fc6dec52ae58817c1b
 size 4999827718

pytorch_model-00003-of-00004.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0b0669b36ff3a762555ac39b4edd3242fb42ff73bb81c366fed0b7b802a44596
+oid sha256:f4b8c712b5b9ea00c96b8a98e313d79f70e1b5d86c89d3c4a64496f4daa1ede6
 size 4915940170

pytorch_model-00004-of-00004.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5337e1c352bea82087ced3063c1df19dd6dd69fc631d9aaaa4e296cadcfb6e0b
+oid sha256:40ba244f3cacf65d554a5a7d04142426f36a6cf75f47394eef6f9f50739cb4f0
 size 1168140873