# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
## Task
Question-answer model in SQUAD dataset

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** Tô Hoàng Minh Tiến
- **Finetuned from model :** bert-mini


<!-- Provide the basic links for the model. -->


## How to Get Started with the Model

Use the code below to get started with the model.

```python
# Load model directly
from transformers import AutoTokenizer, TFAutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("Tien-THM/bert-mini-fine-tuning-squad")
model = TFAutoModelForQuestionAnswering.from_pretrained("Tien-THM/bert-mini-fine-tuning-squad")

import numpy as np

def Inference(context, question):
  encoding = tokenizer(context, question, return_tensors='tf')
  start_pos = model(encoding).start_logits
  end_pos = model(encoding).end_logits
  s = np.argmax(start_pos[0])
  e = np.argmax(end_pos[0])
  print(tokenizer.decode(encoding['input_ids'][0][s:e+1]))

question = 'How many layes does BERT-large have'
context = 'BERT-large is really big... it has 24-layers and an embedding size 
of 1,024, for a total of 340M parameters! Altogether it is 1.34GB, so 
expect it to take a couple minutes to download to your Colab instance'

Inference(context, question)
# Answer: 24 - layers and an em ##bed ##ding size of 1 , 02 ##4
```

## Training Details

### Training Data

Using 2 datasets:
* SQUAD

### Training Procedure 

#### Optimization:
* Adam

#### Loss function
* Cross entropy

#### Training Hyperparameters
* Learning rate: 2e-5
* Batch size: 8
* Epoch: 4
#### Training Loss
| Epoch | Train loss | Validation loss | Exact Match |
|----------|----------|----------|----------|
| #1 | 4.7110 | 3.6251 | 0.38 |
| #2 | 3.2650 | 3.3062 | 0.42 |
| #3 | 2.7899 | 3.2184 | 0.44 |
| #4 | 2.4633 | 3.1946 | 0.45 |



## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Metrics

* Exact Match: 0.45





{"language": ["vi"], "license": "mit", "library_name": "transformers", "metrics": ["exact_match"], "pipeline_tag": "question-answering"}
transformers tf bert question-answering vi license:mit endpoints_compatible region:us