pytorch gpt2 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 1.0
- `alpha` = 2.0
- `beta` = 1.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|7.389|0.892|1.0|
|6.753|0.925|2.0|
|6.567|0.927|3.0|
|6.641|0.926|4.0|
|6.818|0.928|5.0|
|6.492|0.934|6.0|
|6.386|0.932|7.0|
|6.869|0.934|8.0|
|6.730|0.928|9.0|
|6.381|0.927|10.0|
|6.308|0.935|11.0|
|6.305|0.930|12.0|
|6.592|0.931|13.0|
|6.415|0.934|14.0|
|6.477|0.921|15.0|
|6.270|0.933|16.0|
|6.311|0.932|17.0|
|6.255|0.936|18.0|
|6.412|0.935|19.0|
|6.298|0.934|20.0|
|6.390|0.938|21.0|
|6.478|0.933|22.0|
|6.534|0.936|23.0|
|6.919|0.936|24.0|
|6.374|0.933|25.0|
|6.357|0.937|26.0|
|6.519|0.933|27.0|
|6.533|0.926|28.0|
|6.357|0.934|29.0|
|6.197|0.934|30.0|
|6.516|0.934|31.0|
|6.393|0.933|32.0|
|6.099|0.935|33.0|
|6.154|0.936|34.0|
|6.296|0.935|35.0|
|6.204|0.934|36.0|
|6.188|0.938|37.0|
|6.428|0.939|38.0|
|6.397|0.939|39.0|
|6.414|0.940|40.0|
|6.283|0.939|41.0|
|6.351|0.938|42.0|
|6.249|0.940|43.0|
|6.205|0.940|44.0|
|6.125|0.939|45.0|
|6.194|0.941|46.0|
|6.147|0.941|47.0|
|6.219|0.940|48.0|
|6.179|0.941|49.0|
