
__Code availability__
- [shiulab/plantbert](https://github.com/ShiuLab/plantbert)

__Relevant settings__
- env:
  - rand_seed : 20231002
  - test_size : 0.1
  - bert_config:
  - vocab_size : 30_522
  - max_length : 512
- training_arguments:
  - resume_from_ckpt : False
  - mlm : True
  - mlm_prob : 0.2
  - eval_strategy : 'steps'
  - overwrite_out : True
  - num_epochs : 30
  - train_batch_size : 20
  - grad_acc_steps : 8
  - eval_batch_size : 64
  - logging_strategy : 'steps'
  - log_steps : 500
  - save_strategy : 'steps'
  - save_steps : 500
  - safetensors : True
  - load_best_at_end : True
  - save_total_limit : 5

__Model evaluation (loss)__
- Training: 1.8376
- Validation: 1.7476
{"language": ["en"], "license": "mit", "tags": ["biology"], "datasets": ["shius/plantbert-vanilla-cased"]}
transformers pytorch bert fill-mask biology en dataset:shius/plantbert-vanilla-cased license:mit autotrain_compatible endpoints_compatible region:us