pytorch gpt2 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.01
- `alpha` = 2.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|7.057|0.872|1.0|
|6.308|0.919|2.0|
|6.182|0.927|3.0|
|6.121|0.930|4.0|
|6.134|0.930|5.0|
|6.113|0.931|6.0|
|6.074|0.934|7.0|
|6.085|0.933|8.0|
|6.101|0.932|9.0|
|6.024|0.937|10.0|
|6.173|0.927|11.0|
|6.043|0.936|12.0|
|6.049|0.935|13.0|
|6.329|0.918|14.0|
|6.098|0.932|15.0|
|6.040|0.936|16.0|
|6.024|0.937|17.0|
|6.115|0.931|18.0|
|6.112|0.932|19.0|
|6.019|0.937|20.0|
|6.019|0.937|21.0|
|5.997|0.939|22.0|
|6.016|0.938|23.0|
|6.026|0.937|24.0|
|5.959|0.941|25.0|
|5.993|0.939|26.0|
|6.022|0.937|27.0|
|6.003|0.939|28.0|
|5.991|0.939|29.0|
|6.105|0.932|30.0|
|5.967|0.940|31.0|
|5.983|0.940|32.0|
|5.997|0.939|33.0|
|6.007|0.938|34.0|
|5.971|0.940|35.0|
|5.988|0.939|36.0|
|5.957|0.941|37.0|
|5.965|0.941|38.0|
|5.957|0.941|39.0|
|5.977|0.940|40.0|
|5.985|0.940|41.0|
|5.967|0.940|42.0|
|5.992|0.939|43.0|
|5.949|0.941|44.0|
|5.961|0.941|45.0|
|5.968|0.941|46.0|
|5.964|0.941|47.0|
|5.956|0.941|48.0|
|5.963|0.941|49.0|
