pytorch bert en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.01
- `alpha` = 2.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|7.262|0.860|1.0|
|6.747|0.892|2.0|
|7.280|0.859|3.0|
|7.021|0.875|4.0|
|7.106|0.868|5.0|
|7.054|0.873|6.0|
|7.037|0.874|7.0|
|7.054|0.872|8.0|
|7.084|0.871|9.0|
|7.320|0.857|10.0|
|7.094|0.871|11.0|
|7.046|0.874|12.0|
|7.073|0.872|13.0|
|7.151|0.867|14.0|
|7.054|0.874|15.0|
|7.125|0.869|16.0|
|7.034|0.875|17.0|
|7.069|0.872|18.0|
|7.010|0.876|19.0|
|7.048|0.874|20.0|
|6.984|0.878|21.0|
|7.022|0.875|22.0|
|7.020|0.874|23.0|
|7.099|0.871|24.0|
|7.102|0.870|25.0|
|7.797|0.828|26.0|
|7.066|0.873|27.0|
|7.046|0.874|28.0|
|7.104|0.870|29.0|
|7.118|0.870|30.0|
|7.270|0.860|31.0|
|7.103|0.870|32.0|
|7.087|0.872|33.0|
|7.103|0.871|34.0|
|7.941|0.817|35.0|
|7.066|0.873|36.0|
|7.140|0.869|37.0|
|7.055|0.873|38.0|
|7.151|0.868|39.0|
|7.231|0.863|40.0|
|7.158|0.868|41.0|
|7.073|0.873|42.0|
|7.087|0.871|43.0|
|7.108|0.871|44.0|
|7.053|0.873|45.0|
|7.057|0.873|46.0|
|7.075|0.873|47.0|
|7.069|0.873|48.0|
|7.097|0.871|49.0|
