
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-squad

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1638

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 3.0007        | 0.09  | 500   | 1.9956          |
| 1.8347        | 0.18  | 1000  | 1.5820          |
| 1.5656        | 0.27  | 1500  | 1.4727          |
| 1.4915        | 0.36  | 2000  | 1.3595          |
| 1.4139        | 0.45  | 2500  | 1.3064          |
| 1.3317        | 0.54  | 3000  | 1.2853          |
| 1.3158        | 0.63  | 3500  | 1.2326          |
| 1.2748        | 0.72  | 4000  | 1.2094          |
| 1.272         | 0.81  | 4500  | 1.2216          |
| 1.235         | 0.9   | 5000  | 1.1933          |
| 1.2226        | 0.99  | 5500  | 1.1921          |
| 0.9839        | 1.08  | 6000  | 1.1803          |
| 0.9995        | 1.17  | 6500  | 1.1731          |
| 1.0028        | 1.27  | 7000  | 1.1634          |
| 0.9618        | 1.36  | 7500  | 1.1694          |
| 0.9502        | 1.45  | 8000  | 1.1696          |
| 1.0048        | 1.54  | 8500  | 1.1524          |
| 0.955         | 1.63  | 9000  | 1.1296          |
| 0.9533        | 1.72  | 9500  | 1.1376          |
| 0.9578        | 1.81  | 10000 | 1.1292          |
| 0.9523        | 1.9   | 10500 | 1.1116          |
| 0.923         | 1.99  | 11000 | 1.1380          |
| 0.7885        | 2.08  | 11500 | 1.1590          |
| 0.7805        | 2.17  | 12000 | 1.1766          |
| 0.7831        | 2.26  | 12500 | 1.1815          |
| 0.7634        | 2.35  | 13000 | 1.1497          |
| 0.7597        | 2.44  | 13500 | 1.1620          |
| 0.7793        | 2.53  | 14000 | 1.1692          |
| 0.7654        | 2.62  | 14500 | 1.1610          |
| 0.7834        | 2.71  | 15000 | 1.1684          |
| 0.758         | 2.8   | 15500 | 1.1615          |
| 0.7633        | 2.89  | 16000 | 1.1554          |
| 0.7136        | 2.98  | 16500 | 1.1638          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "base_model": "distilbert-base-uncased", "model-index": [{"name": "distilbert-base-uncased-finetuned-squad", "results": []}]}
transformers tensorboard safetensors distilbert question-answering generated_from_trainer base_model:distilbert-base-uncased license:apache-2.0 endpoints_compatible region:us