pytorch en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Dataset: GLUE

Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 2.0
- `beta` = 1.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|16.556|0.800|1.0|
|16.658|0.794|2.0|
|16.329|0.804|3.0|
|16.252|0.816|4.0|
|16.386|0.808|5.0|
|16.747|0.802|6.0|
|16.614|0.807|7.0|
|16.641|0.808|8.0|
|16.362|0.814|9.0|
|16.559|0.805|10.0|
|16.639|0.802|11.0|
|16.819|0.796|12.0|
|16.648|0.803|13.0|
|17.400|0.780|14.0|
|16.121|0.818|15.0|
|16.481|0.808|16.0|
|16.644|0.801|17.0|
|16.747|0.806|18.0|
|16.386|0.808|19.0|
|16.684|0.802|20.0|
|16.766|0.803|21.0|
|16.543|0.803|22.0|
|16.636|0.803|23.0|
|16.468|0.813|24.0|
|16.653|0.800|25.0|
|17.070|0.788|26.0|
|17.067|0.796|27.0|
|16.857|0.796|28.0|
|16.925|0.795|29.0|
|16.890|0.798|30.0|
|16.594|0.801|31.0|
|16.578|0.800|32.0|
|16.517|0.802|33.0|
|16.529|0.802|34.0|
|16.674|0.803|35.0|
|16.543|0.809|36.0|
|16.659|0.803|37.0|
|16.723|0.803|38.0|
|16.480|0.808|39.0|
|16.410|0.804|40.0|
|16.641|0.803|41.0|
|16.328|0.811|42.0|
|16.270|0.813|43.0|
|16.291|0.812|44.0|
|16.522|0.812|45.0|
|16.395|0.810|46.0|
|16.385|0.811|47.0|
|16.392|0.812|48.0|
|16.401|0.811|49.0|
