pytorch bert en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 1.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|13.927|0.888|1.0|
|12.801|0.919|2.0|
|13.474|0.902|3.0|
|15.621|0.836|4.0|
|14.249|0.875|5.0|
|15.069|0.849|6.0|
|15.787|0.827|7.0|
|14.876|0.853|8.0|
|15.467|0.836|9.0|
|14.760|0.859|10.0|
|14.527|0.866|11.0|
|14.600|0.863|12.0|
|14.573|0.864|13.0|
|14.808|0.857|14.0|
|14.598|0.863|15.0|
|14.682|0.861|16.0|
|14.436|0.868|17.0|
|14.817|0.856|18.0|
|14.897|0.854|19.0|
|14.498|0.866|20.0|
|14.495|0.866|21.0|
|14.473|0.867|22.0|
|14.777|0.858|23.0|
|15.106|0.848|24.0|
|14.523|0.865|25.0|
|14.853|0.855|26.0|
|14.600|0.863|27.0|
|14.930|0.852|28.0|
|14.741|0.859|29.0|
|15.183|0.845|30.0|
|14.729|0.859|31.0|
|14.666|0.861|32.0|
|15.021|0.851|33.0|
|14.729|0.859|34.0|
|14.664|0.861|35.0|
|14.875|0.855|36.0|
|14.765|0.858|37.0|
|14.766|0.858|38.0|
|14.681|0.861|39.0|
|14.751|0.859|40.0|
|14.778|0.858|41.0|
|14.846|0.856|42.0|
|14.731|0.859|43.0|
|14.707|0.860|44.0|
|14.712|0.860|45.0|
|14.728|0.859|46.0|
|14.833|0.856|47.0|
|14.727|0.859|48.0|
|14.738|0.859|49.0|
