distilbert_lda_100_v1_book

This model is a fine-tuned version of on the gokulsrinivasagan/processed_book_corpus-ld-100 dataset. It achieves the following results on the evaluation set:

  • Loss: 4.6471
  • Accuracy: 0.7269

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 96
  • eval_batch_size: 96
  • seed: 10
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 10000
  • num_epochs: 25

Training results

Training Loss Epoch Step Validation Loss Accuracy
9.8243 0.4215 10000 9.5492 0.1639
6.4155 0.8431 20000 6.0168 0.5575
5.9278 1.2646 30000 5.5919 0.6085
5.7301 1.6861 40000 5.4051 0.6304
5.5965 2.1077 50000 5.2807 0.6431
5.5012 2.5292 60000 5.1835 0.6540
5.4266 2.9507 70000 5.1245 0.6611
5.3773 3.3723 80000 5.0742 0.6676
5.3364 3.7938 90000 5.0321 0.6726
5.2973 4.2153 100000 5.0044 0.6767
5.2724 4.6369 110000 4.9772 0.6799
5.2442 5.0584 120000 4.9517 0.6836
5.2231 5.4799 130000 4.9291 0.6863
5.2074 5.9014 140000 4.9105 0.6888
5.1812 6.3230 150000 4.8956 0.6911
5.1733 6.7445 160000 4.8813 0.6934
5.147 7.1660 170000 4.8666 0.6953
5.1368 7.5876 180000 4.8567 0.6967
5.1244 8.0091 190000 4.8440 0.6982
5.1142 8.4306 200000 4.8315 0.6998
5.1017 8.8522 210000 4.8245 0.7013
5.0955 9.2737 220000 4.8129 0.7025
5.0784 9.6952 230000 4.8042 0.7039
5.0662 10.1168 240000 4.7974 0.7053
5.067 10.5383 250000 4.7871 0.7062
5.0545 10.9598 260000 4.7792 0.7074
5.0461 11.3814 270000 4.7762 0.7082
5.0456 11.8029 280000 4.7663 0.7093
5.0294 12.2244 290000 4.7599 0.7103
5.0258 12.6460 300000 4.7528 0.7113
5.0149 13.0675 310000 4.7464 0.7123
5.0114 13.4890 320000 4.7420 0.7131
5.0086 13.9106 330000 4.7378 0.7137
5.004 14.3321 340000 4.7310 0.7147
4.9941 14.7536 350000 4.7263 0.7152
4.9902 15.1751 360000 4.7222 0.7157
4.9867 15.5967 370000 4.7158 0.7168
4.9796 16.0182 380000 4.7116 0.7175
4.9751 16.4397 390000 4.7051 0.7180
4.9683 16.8613 400000 4.7038 0.7184
4.967 17.2828 410000 4.6955 0.7196
4.961 17.7043 420000 4.6947 0.7200
4.953 18.1259 430000 4.6910 0.7204
4.9491 18.5474 440000 4.6884 0.7208
4.9485 18.9689 450000 4.6825 0.7217
4.9439 19.3905 460000 4.6790 0.7222
4.9417 19.8120 470000 4.6757 0.7226
4.9334 20.2335 480000 4.6713 0.7233
4.929 20.6551 490000 4.6686 0.7238
4.925 21.0766 500000 4.6645 0.7242
4.9207 21.4981 510000 4.6618 0.7246
4.9177 21.9197 520000 4.6599 0.7250
4.9191 22.3412 530000 4.6584 0.7252
4.9138 22.7627 540000 4.6577 0.7255
4.9139 23.1843 550000 4.6533 0.7259
4.9098 23.6058 560000 4.6508 0.7264
4.9063 24.0273 570000 4.6497 0.7265
4.9048 24.4488 580000 4.6457 0.7271
4.9011 24.8704 590000 4.6463 0.7270

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.2.1+cu118
  • Datasets 2.17.0
  • Tokenizers 0.20.3
Downloads last month
4
Safetensors
Model size
67.7M params
Tensor type
F32
·
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for gokulsrinivasagan/distilbert_lda_100_v1_book

Finetunes
9 models

Dataset used to train gokulsrinivasagan/distilbert_lda_100_v1_book

Evaluation results

  • Accuracy on gokulsrinivasagan/processed_book_corpus-ld-100
    self-reported
    0.727