
# bert-paper-classifier

This model is a fine-tuned version of [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract) on [the dataset](https://www.kaggle.com/datasets/neelshah18/arxivdataset) with titles and abstracts of papers from arXiv.

## Intended uses & limitations

This model is intended to predict the [arXiv taxonomy terms](https://arxiv.org/category_taxonomy) given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict `cs.CL` (computation and language) as the mostl likely category for [the original transformers paper](https://arxiv.org/abs/1706.03762).

## Training procedure

The code for the model fine-tuning can be found [in the respective notebook](https://huggingface.co/oracat/bert-paper-classifier-arxiv/blob/main/finetuning-arxiv.ipynb).

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3
{"license": "mit", "tags": ["generated_from_trainer"], "metrics": ["accuracy"], "model-index": [{"name": "bert-paper-classifier", "results": []}]}
transformers pytorch bert text-classification generated_from_trainer arxiv:1706.03762 license:mit autotrain_compatible endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #bert #text-classification #generated_from_trainer #arxiv-1706.03762 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 


# bert-paper-classifier

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract on the dataset with titles and abstracts of papers from arXiv.

## Intended uses & limitations

This model is intended to predict the arXiv taxonomy terms given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict 'cs.CL' (computation and language) as the mostl likely category for the original transformers paper.

## Training procedure

The code for the model fine-tuning can be found in the respective notebook.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3
# bert-paper-classifier

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract on the dataset with titles and abstracts of papers from arXiv. ## Intended uses & limitations

This model is intended to predict the arXiv taxonomy terms given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict 'cs.CL' (computation and language) as the mostl likely category for the original transformers paper. ## Training procedure

The code for the model fine-tuning can be found in the respective notebook. ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3 ### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3
TAGS
#transformers #pytorch #bert #text-classification #generated_from_trainer #arxiv-1706.03762 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 
 # bert-paper-classifier

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract on the dataset with titles and abstracts of papers from arXiv. ## Intended uses & limitations

This model is intended to predict the arXiv taxonomy terms given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict 'cs.CL' (computation and language) as the mostl likely category for the original transformers paper. ## Training procedure

The code for the model fine-tuning can be found in the respective notebook. ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3 ### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3
TAGS
#transformers #pytorch #bert #text-classification #generated_from_trainer #arxiv-1706.03762 #license-mit #autotrain_compatible #endpoints_compatible #has_space #region-us 
# bert-paper-classifier

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract on the dataset with titles and abstracts of papers from arXiv.## Intended uses & limitations

This model is intended to predict the arXiv taxonomy terms given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict 'cs.CL' (computation and language) as the mostl likely category for the original transformers paper.## Training procedure

The code for the model fine-tuning can be found in the respective notebook.### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 128
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3