
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# biomedical_question_answering

This model is a fine-tuned version of [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext) on a custom dataset of question answer pairs annotated from research papers from Pubmed.
It achieves the following results on the evaluation set:
- Loss: 2.6629

## Model description

Model finetuned on PubmedBERT using custom daatset

## Intended uses & limitations

For question answering related to biomedical research papers. 

## Training and evaluation data

Data https://huggingface.co/datasets/Shushant/BiomedicalQuestionAnsweringDataset 

## Training procedure

Finetuning using Trainer API


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 236  | 1.6866          |
| No log        | 2.0   | 472  | 1.5432          |
| 0.737         | 3.0   | 708  | 1.7998          |
| 0.737         | 4.0   | 944  | 1.9746          |
| 0.2893        | 5.0   | 1180 | 1.9510          |
| 0.2893        | 6.0   | 1416 | 2.1479          |
| 0.1562        | 7.0   | 1652 | 2.3304          |
| 0.1562        | 8.0   | 1888 | 2.5882          |
| 0.0823        | 9.0   | 2124 | 2.6494          |
| 0.0823        | 10.0  | 2360 | 2.6629          |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.9.0
- Tokenizers 0.13.2

- # Paper Details
- If you want to know about the full implementation detials, please read the full paper here
  https://www.researchgate.net/publication/375011546_Question_Answering_on_Biomedical_Research_Papers_using_Transfer_Learning_on_BERT-Base_Models

  ## Citation Plain Text
  S. Pudasaini and S. Shakya, "Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models," 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Kirtipur, Nepal, 2023, pp. 496-501, doi: 10.1109/I-SMAC58438.2023.10290240.

  ## Citation Bibtex
@INPROCEEDINGS{10290240,
  author={Pudasaini, Shushanta and Shakya, Subarna},
  booktitle={2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)}, 
  title={Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models}, 
  year={2023},
  volume={},
  number={},
  pages={496-501},
  doi={10.1109/I-SMAC58438.2023.10290240}}

  
{"language": ["en"], "license": "mit", "library_name": "transformers", "tags": ["generated_from_trainer"], "datasets": ["Shushant/BiomedicalQuestionAnsweringDataset"], "metrics": ["exact_match", "f1"], "pipeline_tag": "question-answering", "model-index": [{"name": "biomedical_question_answering", "results": []}]}
transformers pytorch tensorboard bert question-answering generated_from_trainer en dataset:Shushant/BiomedicalQuestionAnsweringDataset license:mit endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #bert #question-answering #generated_from_trainer #en #dataset-Shushant/BiomedicalQuestionAnsweringDataset #license-mit #endpoints_compatible #region-us 

biomedical\_question\_answering
===============================


This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext on a custom dataset of question answer pairs annotated from research papers from Pubmed.
It achieves the following results on the evaluation set:


* Loss: 2.6629


Model description
-----------------


Model finetuned on PubmedBERT using custom daatset


Intended uses & limitations
---------------------------


For question answering related to biomedical research papers.


Training and evaluation data
----------------------------


Data URL


Training procedure
------------------


Finetuning using Trainer API


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 3e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 10


### Training results



### Framework versions


* Transformers 4.26.1
* Pytorch 1.13.1+cu116
* Datasets 2.9.0
* Tokenizers 0.13.2
* Paper Details
=============
* If you want to know about the full implementation detials, please read the full paper here
URL


Plain Text
S. Pudasaini and S. Shakya, "Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models," 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Kirtipur, Nepal, 2023, pp. 496-501, doi: 10.1109/I-SMAC58438.2023.10290240.


Bibtex
@INPROCEEDINGS{10290240,
author={Pudasaini, Shushanta and Shakya, Subarna},
booktitle={2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)},
title={Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models},
year={2023},
volume={},
number={},
pages={496-501},
doi={10.1109/I-SMAC58438.2023.10290240}}



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 3e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 10 ### Training results ### Framework versions


* Transformers 4.26.1
* Pytorch 1.13.1+cu116
* Datasets 2.9.0
* Tokenizers 0.13.2
* Paper Details
=============
* If you want to know about the full implementation detials, please read the full paper here
URL


Plain Text
S. Pudasaini and S. Shakya, "Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models," 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Kirtipur, Nepal, 2023, pp. 496-501, doi: 10.1109/I-SMAC58438.2023.10290240.


Bibtex
@INPROCEEDINGS{10290240,
author={Pudasaini, Shushanta and Shakya, Subarna},
booktitle={2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)},
title={Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models},
year={2023},
volume={},
number={},
pages={496-501},
doi={10.1109/I-SMAC58438.2023.10290240}}
TAGS
#transformers #pytorch #tensorboard #bert #question-answering #generated_from_trainer #en #dataset-Shushant/BiomedicalQuestionAnsweringDataset #license-mit #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 3e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 10 ### Training results ### Framework versions


* Transformers 4.26.1
* Pytorch 1.13.1+cu116
* Datasets 2.9.0
* Tokenizers 0.13.2
* Paper Details
=============
* If you want to know about the full implementation detials, please read the full paper here
URL


Plain Text
S. Pudasaini and S. Shakya, "Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models," 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Kirtipur, Nepal, 2023, pp. 496-501, doi: 10.1109/I-SMAC58438.2023.10290240.


Bibtex
@INPROCEEDINGS{10290240,
author={Pudasaini, Shushanta and Shakya, Subarna},
booktitle={2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)},
title={Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models},
year={2023},
volume={},
number={},
pages={496-501},
doi={10.1109/I-SMAC58438.2023.10290240}}
TAGS
#transformers #pytorch #tensorboard #bert #question-answering #generated_from_trainer #en #dataset-Shushant/BiomedicalQuestionAnsweringDataset #license-mit #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 3e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 10### Training results### Framework versions


* Transformers 4.26.1
* Pytorch 1.13.1+cu116
* Datasets 2.9.0
* Tokenizers 0.13.2
* Paper Details
=============
* If you want to know about the full implementation detials, please read the full paper here
URL


Plain Text
S. Pudasaini and S. Shakya, "Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models," 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Kirtipur, Nepal, 2023, pp. 496-501, doi: 10.1109/I-SMAC58438.2023.10290240.


Bibtex
@INPROCEEDINGS{10290240,
author={Pudasaini, Shushanta and Shakya, Subarna},
booktitle={2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)},
title={Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models},
year={2023},
volume={},
number={},
pages={496-501},
doi={10.1109/I-SMAC58438.2023.10290240}}