pytorch t5 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 2.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|11.926|0.939|1.0|
|11.887|0.941|2.0|
|11.913|0.941|3.0|
|11.912|0.940|4.0|
|11.878|0.942|5.0|
|11.784|0.945|6.0|
|11.718|0.947|7.0|
|11.742|0.946|8.0|
|11.763|0.945|9.0|
|11.736|0.946|10.0|
|11.728|0.946|11.0|
|11.790|0.944|12.0|
|11.811|0.944|13.0|
|11.784|0.945|14.0|
|11.894|0.941|15.0|
|11.739|0.946|16.0|
|11.722|0.947|17.0|
|11.959|0.940|18.0|
|11.813|0.944|19.0|
|11.733|0.946|20.0|
|11.687|0.948|21.0|
|11.707|0.947|22.0|
|11.700|0.947|23.0|
|11.691|0.948|24.0|
|11.827|0.943|25.0|
|11.720|0.947|26.0|
|11.698|0.947|27.0|
|11.692|0.948|28.0|
|11.707|0.947|29.0|
|11.706|0.947|30.0|
|11.711|0.947|31.0|
|11.721|0.947|32.0|
|11.687|0.948|33.0|
|11.728|0.946|34.0|
|11.678|0.948|35.0|
|11.691|0.948|36.0|
|11.692|0.948|37.0|
|11.684|0.948|38.0|
|11.743|0.946|39.0|
|11.631|0.950|40.0|
|11.707|0.947|41.0|
|11.652|0.949|42.0|
|11.688|0.948|43.0|
|11.647|0.949|44.0|
|11.639|0.949|45.0|
|11.660|0.949|46.0|
|11.652|0.949|47.0|
|11.648|0.949|48.0|
|11.652|0.949|49.0|
