pytorch gpt2 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.0001
- `alpha` = 2.0
- `beta` = 1.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|12.577|0.916|1.0|
|14.443|0.857|2.0|
|12.267|0.928|3.0|
|12.056|0.934|4.0|
|12.092|0.934|5.0|
|12.172|0.932|6.0|
|12.208|0.930|7.0|
|12.934|0.908|8.0|
|11.949|0.939|9.0|
|12.090|0.935|10.0|
|12.315|0.927|11.0|
|12.018|0.937|12.0|
|12.001|0.938|13.0|
|12.128|0.934|14.0|
|12.064|0.935|15.0|
|11.960|0.938|16.0|
|12.116|0.934|17.0|
|12.313|0.928|18.0|
|11.999|0.938|19.0|
|11.996|0.938|20.0|
|12.019|0.937|21.0|
|12.002|0.938|22.0|
|12.180|0.932|23.0|
|11.963|0.939|24.0|
|12.045|0.936|25.0|
|11.942|0.940|26.0|
|12.153|0.933|27.0|
|11.840|0.943|28.0|
|11.905|0.941|29.0|
|11.899|0.941|30.0|
|11.854|0.942|31.0|
|11.972|0.939|32.0|
|11.922|0.940|33.0|
|11.922|0.940|34.0|
|11.894|0.941|35.0|
|11.878|0.942|36.0|
|11.853|0.942|37.0|
|11.914|0.940|38.0|
|11.867|0.942|39.0|
|11.862|0.942|40.0|
|11.855|0.942|41.0|
|11.900|0.941|42.0|
|11.828|0.943|43.0|
|11.853|0.942|44.0|
|11.847|0.943|45.0|
|11.867|0.942|46.0|
|11.868|0.942|47.0|
|11.838|0.943|48.0|
|11.832|0.943|49.0|
