tensorboard safetensors generated_from_trainer base_model:indolem/indobert-base-uncased license:mit region:us
---
license: mit
base_model: indolem/indobert-base-uncased
tags:
- generated_from_trainer
model-index:
- name: QA_using_indoBERT_LORA_qv2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# QA_using_indoBERT_LORA_qv2

This model is a fine-tuned version of [indolem/indobert-base-uncased](https://huggingface.co/indolem/indobert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9435

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 5.7426        | 0.02  | 500   | 6.2378          |
| 5.1601        | 0.03  | 1000  | 4.0267          |
| 3.466         | 0.05  | 1500  | 3.0399          |
| 2.9304        | 0.06  | 2000  | 2.8011          |
| 2.7403        | 0.08  | 2500  | 2.7113          |
| 2.599         | 0.09  | 3000  | 2.6337          |
| 2.4993        | 0.11  | 3500  | 2.4798          |
| 2.4454        | 0.12  | 4000  | 2.4486          |
| 2.3938        | 0.14  | 4500  | 2.3848          |
| 2.3124        | 0.15  | 5000  | 2.3729          |
| 2.2595        | 0.17  | 5500  | 2.4021          |
| 2.241         | 0.18  | 6000  | 2.3487          |
| 2.3296        | 0.2   | 6500  | 2.2819          |
| 2.21          | 0.21  | 7000  | 2.2588          |
| 2.2386        | 0.23  | 7500  | 2.3498          |
| 2.164         | 0.25  | 8000  | 2.2315          |
| 2.2535        | 0.26  | 8500  | 2.2315          |
| 2.2621        | 0.28  | 9000  | 2.3788          |
| 2.364         | 0.29  | 9500  | 2.8077          |
| 2.2345        | 0.31  | 10000 | 2.2495          |
| 2.1571        | 0.32  | 10500 | 2.2306          |
| 2.0452        | 0.34  | 11000 | 2.2417          |
| 2.1279        | 0.35  | 11500 | 2.1814          |
| 2.1482        | 0.37  | 12000 | 2.1762          |
| 2.1064        | 0.38  | 12500 | 2.1931          |
| 1.9992        | 0.4   | 13000 | 2.1902          |
| 2.1265        | 0.41  | 13500 | 2.1558          |
| 2.0659        | 0.43  | 14000 | 2.2007          |
| 2.0314        | 0.44  | 14500 | 2.1326          |
| 2.0086        | 0.46  | 15000 | 2.1282          |
| 2.0168        | 0.48  | 15500 | 2.1372          |
| 2.024         | 0.49  | 16000 | 2.1111          |
| 2.0636        | 0.51  | 16500 | 2.0926          |
| 1.9673        | 0.52  | 17000 | 2.1200          |
| 2.0207        | 0.54  | 17500 | 2.1710          |
| 2.0857        | 0.55  | 18000 | 2.1886          |
| 2.1617        | 0.57  | 18500 | 2.1123          |
| 1.9912        | 0.58  | 19000 | 2.0999          |
| 2.1166        | 0.6   | 19500 | 2.0940          |
| 2.0312        | 0.61  | 20000 | 2.1436          |
| 2.1124        | 0.63  | 20500 | 2.1743          |
| 2.0399        | 0.64  | 21000 | 2.0801          |
| 1.9246        | 0.66  | 21500 | 2.0535          |
| 1.9792        | 0.67  | 22000 | 2.0926          |
| 1.9713        | 0.69  | 22500 | 2.0666          |
| 1.9285        | 0.71  | 23000 | 2.0699          |
| 1.9454        | 0.72  | 23500 | 2.0873          |
| 1.9255        | 0.74  | 24000 | 2.0515          |
| 1.9428        | 0.75  | 24500 | 2.0771          |
| 1.9093        | 0.77  | 25000 | 2.0538          |
| 1.933         | 0.78  | 25500 | 2.0308          |
| 1.8628        | 0.8   | 26000 | 2.0554          |
| 1.906         | 0.81  | 26500 | 2.0581          |
| 1.9255        | 0.83  | 27000 | 2.0167          |
| 1.8795        | 0.84  | 27500 | 2.0423          |
| 1.8987        | 0.86  | 28000 | 2.0300          |
| 1.8464        | 0.87  | 28500 | 2.0540          |
| 1.9619        | 0.89  | 29000 | 2.0068          |
| 1.9475        | 0.9   | 29500 | 2.0079          |
| 1.9399        | 0.92  | 30000 | 1.9889          |
| 1.8473        | 0.94  | 30500 | 2.0135          |
| 1.8775        | 0.95  | 31000 | 2.0096          |
| 1.8049        | 0.97  | 31500 | 2.0289          |
| 1.8029        | 0.98  | 32000 | 2.0561          |
| 1.9167        | 1.0   | 32500 | 2.0199          |
| 1.873         | 1.01  | 33000 | 2.0081          |
| 1.7915        | 1.03  | 33500 | 2.0418          |
| 1.8741        | 1.04  | 34000 | 2.0087          |
| 1.8528        | 1.06  | 34500 | 2.0023          |
| 1.8255        | 1.07  | 35000 | 2.0275          |
| 1.8667        | 1.09  | 35500 | 2.0227          |
| 1.7821        | 1.1   | 36000 | 1.9990          |
| 1.7809        | 1.12  | 36500 | 2.0067          |
| 1.8287        | 1.13  | 37000 | 1.9984          |
| 1.8026        | 1.15  | 37500 | 2.0272          |
| 1.8299        | 1.16  | 38000 | 2.0259          |
| 1.7972        | 1.18  | 38500 | 2.0382          |
| 1.8505        | 1.2   | 39000 | 1.9803          |
| 1.8319        | 1.21  | 39500 | 1.9699          |
| 1.8171        | 1.23  | 40000 | 1.9931          |
| 1.7986        | 1.24  | 40500 | 1.9933          |
| 1.8228        | 1.26  | 41000 | 1.9807          |
| 1.8793        | 1.27  | 41500 | 1.9999          |
| 1.7724        | 1.29  | 42000 | 1.9779          |
| 1.7328        | 1.3   | 42500 | 1.9725          |
| 1.8083        | 1.32  | 43000 | 1.9603          |
| 1.7829        | 1.33  | 43500 | 1.9790          |
| 1.7823        | 1.35  | 44000 | 1.9777          |
| 1.7715        | 1.36  | 44500 | 1.9831          |
| 1.8368        | 1.38  | 45000 | 1.9531          |
| 1.7688        | 1.39  | 45500 | 1.9666          |
| 1.7946        | 1.41  | 46000 | 1.9662          |
| 1.8104        | 1.43  | 46500 | 1.9799          |
| 1.758         | 1.44  | 47000 | 1.9697          |
| 1.802         | 1.46  | 47500 | 1.9617          |
| 1.7628        | 1.47  | 48000 | 1.9645          |
| 1.8014        | 1.49  | 48500 | 1.9642          |
| 1.8153        | 1.5   | 49000 | 1.9449          |
| 1.7997        | 1.52  | 49500 | 1.9682          |
| 1.8021        | 1.53  | 50000 | 1.9567          |
| 1.766         | 1.55  | 50500 | 1.9740          |
| 1.7886        | 1.56  | 51000 | 1.9513          |
| 1.7865        | 1.58  | 51500 | 1.9411          |
| 1.8403        | 1.59  | 52000 | 1.9396          |
| 1.7257        | 1.61  | 52500 | 1.9590          |
| 1.7743        | 1.62  | 53000 | 1.9408          |
| 1.7903        | 1.64  | 53500 | 1.9469          |
| 1.8302        | 1.66  | 54000 | 1.9370          |
| 1.7979        | 1.67  | 54500 | 1.9394          |
| 1.8109        | 1.69  | 55000 | 1.9440          |
| 1.7397        | 1.7   | 55500 | 1.9579          |
| 1.7374        | 1.72  | 56000 | 1.9501          |
| 1.7373        | 1.73  | 56500 | 1.9518          |
| 1.7273        | 1.75  | 57000 | 1.9474          |
| 1.8064        | 1.76  | 57500 | 1.9368          |
| 1.7913        | 1.78  | 58000 | 1.9426          |
| 1.8166        | 1.79  | 58500 | 1.9331          |
| 1.8238        | 1.81  | 59000 | 1.9341          |
| 1.8049        | 1.82  | 59500 | 1.9464          |
| 1.8735        | 1.84  | 60000 | 1.9397          |
| 1.8169        | 1.85  | 60500 | 1.9388          |
| 1.7689        | 1.87  | 61000 | 1.9393          |
| 1.7612        | 1.89  | 61500 | 1.9433          |
| 1.7768        | 1.9   | 62000 | 1.9402          |
| 1.6952        | 1.92  | 62500 | 1.9478          |
| 1.7951        | 1.93  | 63000 | 1.9395          |
| 1.764         | 1.95  | 63500 | 1.9381          |
| 1.7895        | 1.96  | 64000 | 1.9362          |
| 1.6671        | 1.98  | 64500 | 1.9428          |
| 1.7535        | 1.99  | 65000 | 1.9435          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
