climate

This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on the climate_1_day dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4520

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 12
  • eval_batch_size: 12
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • total_train_batch_size: 24
  • total_eval_batch_size: 24
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
1.8708 0.0267 10 1.7567
1.7168 0.0533 20 1.6884
1.6815 0.08 30 1.6499
1.6333 0.1067 40 1.6236
1.6028 0.1333 50 1.6039
1.5665 0.16 60 1.5895
1.574 0.1867 70 1.5772
1.5862 0.2133 80 1.5648
1.5218 0.24 90 1.5564
1.5451 0.2667 100 1.5466
1.5011 0.2933 110 1.5382
1.5182 0.32 120 1.5328
1.5331 0.3467 130 1.5240
1.5096 0.3733 140 1.5180
1.5433 0.4 150 1.5125
1.4919 0.4267 160 1.5082
1.5119 0.4533 170 1.5037
1.4898 0.48 180 1.4962
1.4879 0.5067 190 1.4910
1.4813 0.5333 200 1.4869
1.4776 0.56 210 1.4821
1.4786 0.5867 220 1.4783
1.4825 0.6133 230 1.4740
1.4525 0.64 240 1.4710
1.4794 0.6667 250 1.4680
1.4785 0.6933 260 1.4655
1.4523 0.72 270 1.4628
1.4618 0.7467 280 1.4605
1.4751 0.7733 290 1.4581
1.4263 0.8 300 1.4561
1.4421 0.8267 310 1.4548
1.4824 0.8533 320 1.4539
1.4675 0.88 330 1.4531
1.4891 0.9067 340 1.4525
1.4617 0.9333 350 1.4520
1.4404 0.96 360 1.4521
1.44 0.9867 370 1.4520

Framework versions

  • PEFT 0.12.0
  • Transformers 4.46.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.21.0
  • Tokenizers 0.20.1
Downloads last month
2
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for Howard881010/climate-1day

Adapter
(536)
this model