peft safetensors generated_from_trainer base_model:outputs/solar_10.7_darulm_unigram_proj_init_8node_darulm_part1_v3_1.0_512_12_02_24 region:us
---
library_name: peft
tags:
- generated_from_trainer
metrics:
- accuracy
base_model: outputs/solar_10.7_darulm_unigram_proj_init_8node_darulm_part1_v3_1.0_512_12_02_24
model-index:
- name: solar_10.7_darulm_unigram_proj_init_darulm_part2_r128_a512_v3_1.0_512_28_02_24
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# solar_10.7_darulm_unigram_proj_init_darulm_part2_r128_a512_v3_1.0_512_28_02_24

This model is a fine-tuned version of [outputs/solar_10.7_darulm_unigram_proj_init_8node_darulm_part1_v3_1.0_512_12_02_24](https://huggingface.co/outputs/solar_10.7_darulm_unigram_proj_init_8node_darulm_part1_v3_1.0_512_12_02_24) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2309
- Accuracy: 0.5309

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 24
- gradient_accumulation_steps: 5
- total_train_batch_size: 120
- total_eval_batch_size: 24
- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05
- lr_scheduler_type: linear
- num_epochs: 1.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| No log        | 0.0   | 1     | 2.3534          | 0.5148   |
| 2.4427        | 0.01  | 500   | 2.3338          | 0.5155   |
| 2.4399        | 0.02  | 1000  | 2.3276          | 0.5164   |
| 2.4244        | 0.03  | 1500  | 2.3231          | 0.5169   |
| 2.4336        | 0.04  | 2000  | 2.3194          | 0.5177   |
| 2.4201        | 0.06  | 2500  | 2.3156          | 0.5180   |
| 2.4245        | 0.07  | 3000  | 2.3128          | 0.5185   |
| 2.4157        | 0.08  | 3500  | 2.3097          | 0.5187   |
| 2.4054        | 0.09  | 4000  | 2.3070          | 0.5194   |
| 2.4161        | 0.1   | 4500  | 2.3033          | 0.5197   |
| 2.395         | 0.11  | 5000  | 2.3020          | 0.5201   |
| 2.4037        | 0.12  | 5500  | 2.3001          | 0.5204   |
| 2.4188        | 0.13  | 6000  | 2.2977          | 0.5206   |
| 2.406         | 0.15  | 6500  | 2.2961          | 0.5208   |
| 2.4022        | 0.16  | 7000  | 2.2943          | 0.5210   |
| 2.3952        | 0.17  | 7500  | 2.2926          | 0.5217   |
| 2.394         | 0.18  | 8000  | 2.2909          | 0.5217   |
| 2.3828        | 0.19  | 8500  | 2.2891          | 0.5218   |
| 2.3903        | 0.2   | 9000  | 2.2882          | 0.5223   |
| 2.3943        | 0.21  | 9500  | 2.2861          | 0.5224   |
| 2.3944        | 0.22  | 10000 | 2.2851          | 0.5224   |
| 2.3872        | 0.23  | 10500 | 2.2841          | 0.5227   |
| 2.381         | 0.25  | 11000 | 2.2820          | 0.5228   |
| 2.3832        | 0.26  | 11500 | 2.2798          | 0.5232   |
| 2.3813        | 0.27  | 12000 | 2.2793          | 0.5237   |
| 2.3715        | 0.28  | 12500 | 2.2779          | 0.5241   |
| 2.3898        | 0.29  | 13000 | 2.2764          | 0.5240   |
| 2.3717        | 0.3   | 13500 | 2.2757          | 0.5240   |
| 2.3745        | 0.31  | 14000 | 2.2742          | 0.5244   |
| 2.3657        | 0.32  | 14500 | 2.2732          | 0.5244   |
| 2.3782        | 0.34  | 15000 | 2.2715          | 0.5247   |
| 2.3761        | 0.35  | 15500 | 2.2706          | 0.5247   |
| 2.3827        | 0.36  | 16000 | 2.2692          | 0.5249   |
| 2.3659        | 0.37  | 16500 | 2.2678          | 0.5251   |
| 2.3551        | 0.38  | 17000 | 2.2674          | 0.5252   |
| 2.3605        | 0.39  | 17500 | 2.2662          | 0.5255   |
| 2.3579        | 0.4   | 18000 | 2.2654          | 0.5256   |
| 2.361         | 0.41  | 18500 | 2.2642          | 0.5257   |
| 2.3632        | 0.42  | 19000 | 2.2652          | 0.5254   |
| 2.3409        | 0.44  | 19500 | 2.2625          | 0.5261   |
| 2.3546        | 0.45  | 20000 | 2.2631          | 0.5259   |
| 2.361         | 0.46  | 20500 | 2.2611          | 0.5264   |
| 2.355         | 0.47  | 21000 | 2.2598          | 0.5264   |
| 2.3599        | 0.48  | 21500 | 2.2588          | 0.5265   |
| 2.3554        | 0.49  | 22000 | 2.2583          | 0.5265   |
| 2.3552        | 0.5   | 22500 | 2.2571          | 0.5268   |
| 2.3574        | 0.51  | 23000 | 2.2565          | 0.5268   |
| 2.3527        | 0.53  | 23500 | 2.2557          | 0.5272   |
| 2.3574        | 0.54  | 24000 | 2.2548          | 0.5272   |
| 2.3395        | 0.55  | 24500 | 2.2534          | 0.5274   |
| 2.3517        | 0.56  | 25000 | 2.2531          | 0.5272   |
| 2.346         | 0.57  | 25500 | 2.2521          | 0.5275   |
| 2.3469        | 0.58  | 26000 | 2.2515          | 0.5275   |
| 2.3451        | 0.59  | 26500 | 2.2509          | 0.5278   |
| 2.3373        | 0.6   | 27000 | 2.2501          | 0.5277   |
| 2.3512        | 0.61  | 27500 | 2.2493          | 0.5281   |
| 2.3351        | 0.63  | 28000 | 2.2485          | 0.5282   |
| 2.3431        | 0.64  | 28500 | 2.2476          | 0.5282   |
| 2.3399        | 0.65  | 29000 | 2.2463          | 0.5283   |
| 2.3376        | 0.66  | 29500 | 2.2463          | 0.5284   |
| 2.3574        | 0.67  | 30000 | 2.2456          | 0.5285   |
| 2.3312        | 0.68  | 30500 | 2.2447          | 0.5289   |
| 2.3442        | 0.69  | 31000 | 2.2442          | 0.5288   |
| 2.338         | 0.7   | 31500 | 2.2434          | 0.5289   |
| 2.3345        | 0.72  | 32000 | 2.2433          | 0.5291   |
| 2.3314        | 0.73  | 32500 | 2.2420          | 0.5292   |
| 2.326         | 0.74  | 33000 | 2.2414          | 0.5293   |
| 2.3247        | 0.75  | 33500 | 2.2409          | 0.5295   |
| 2.3363        | 0.76  | 34000 | 2.2403          | 0.5296   |
| 2.3409        | 0.77  | 34500 | 2.2395          | 0.5297   |
| 2.335         | 0.78  | 35000 | 2.2391          | 0.5295   |
| 2.3194        | 0.79  | 35500 | 2.2383          | 0.5298   |
| 2.3367        | 0.8   | 36000 | 2.2379          | 0.5301   |
| 2.3286        | 0.82  | 36500 | 2.2372          | 0.5301   |
| 2.3225        | 0.83  | 37000 | 2.2366          | 0.5302   |
| 2.3198        | 0.84  | 37500 | 2.2363          | 0.5301   |
| 2.3274        | 0.85  | 38000 | 2.2355          | 0.5301   |
| 2.3195        | 0.86  | 38500 | 2.2349          | 0.5303   |
| 2.3418        | 0.87  | 39000 | 2.2344          | 0.5303   |
| 2.323         | 0.88  | 39500 | 2.2340          | 0.5304   |
| 2.3211        | 0.89  | 40000 | 2.2336          | 0.5304   |
| 2.3332        | 0.91  | 40500 | 2.2334          | 0.5306   |
| 2.3226        | 0.92  | 41000 | 2.2329          | 0.5307   |
| 2.3329        | 0.93  | 41500 | 2.2325          | 0.5308   |
| 2.3172        | 0.94  | 42000 | 2.2321          | 0.5307   |
| 2.3231        | 0.95  | 42500 | 2.2319          | 0.5308   |
| 2.314         | 0.96  | 43000 | 2.2316          | 0.5309   |
| 2.3205        | 0.97  | 43500 | 2.2315          | 0.5308   |
| 2.3208        | 0.98  | 44000 | 2.2312          | 0.5309   |
| 2.3228        | 0.99  | 44500 | 2.2310          | 0.5309   |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.1.2
- Datasets 2.16.1
- Tokenizers 0.15.2
## Training procedure


### Framework versions


- PEFT 0.6.0
