pytorch en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Dataset: GLUE

Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 2.0
- `beta` = 1.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|21.497|0.578|1.0|
|21.579|0.578|2.0|
|22.797|0.567|3.0|
|23.206|0.563|4.0|
|22.825|0.581|5.0|
|22.042|0.606|6.0|
|23.542|0.556|7.0|
|22.290|0.606|8.0|
|23.461|0.567|9.0|
|23.063|0.585|10.0|
|23.358|0.574|11.0|
|23.223|0.578|12.0|
|23.578|0.567|13.0|
|22.854|0.585|14.0|
|23.723|0.563|15.0|
|23.165|0.574|16.0|
|23.134|0.574|17.0|
|23.479|0.560|18.0|
|22.982|0.581|19.0|
|23.601|0.560|20.0|
|22.914|0.585|21.0|
|22.700|0.599|22.0|
|23.201|0.574|23.0|
|23.437|0.570|24.0|
|23.407|0.578|25.0|
|22.512|0.603|26.0|
|23.252|0.567|27.0|
|23.777|0.556|28.0|
|23.993|0.556|29.0|
|23.708|0.563|30.0|
|23.886|0.552|31.0|
|23.857|0.556|32.0|
|23.593|0.556|33.0|
|23.368|0.570|34.0|
|23.654|0.563|35.0|
|24.380|0.542|36.0|
|24.091|0.549|37.0|
|23.997|0.549|38.0|
|23.977|0.552|39.0|
|23.761|0.560|40.0|
|23.537|0.570|41.0|
|23.381|0.574|42.0|
|23.864|0.556|43.0|
|23.737|0.556|44.0|
|23.261|0.578|45.0|
|23.832|0.560|46.0|
|23.446|0.567|47.0|
|23.397|0.570|48.0|
|23.439|0.570|49.0|
