pytorch t5 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 16
- `gradient_accumulation_steps` = 1
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 2.0
- `beta` = 1.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|8.986|0.938|1.0|
|9.101|0.932|2.0|
|8.912|0.941|3.0|
|8.916|0.941|4.0|
|8.902|0.942|5.0|
|8.814|0.946|6.0|
|8.843|0.945|7.0|
|8.837|0.945|8.0|
|8.830|0.945|9.0|
|8.847|0.944|10.0|
|8.822|0.945|11.0|
|8.808|0.946|12.0|
|8.806|0.946|13.0|
|8.814|0.946|14.0|
|8.807|0.946|15.0|
|8.933|0.941|16.0|
|8.824|0.945|17.0|
|8.840|0.945|18.0|
|8.854|0.944|19.0|
|8.850|0.944|20.0|
|8.901|0.942|21.0|
|8.862|0.944|22.0|
|8.856|0.944|23.0|
|8.870|0.943|24.0|
|8.800|0.946|25.0|
|8.800|0.947|26.0|
|8.825|0.945|27.0|
|8.782|0.947|28.0|
|8.772|0.948|29.0|
|8.767|0.948|30.0|
|8.756|0.948|31.0|
|8.793|0.947|32.0|
|8.738|0.949|33.0|
|8.728|0.949|34.0|
|8.709|0.950|35.0|
|8.694|0.951|36.0|
|8.740|0.949|37.0|
|8.723|0.950|38.0|
|8.710|0.950|39.0|
|8.735|0.949|40.0|
|8.705|0.950|41.0|
|8.699|0.951|42.0|
|8.714|0.950|43.0|
|8.706|0.950|44.0|
|8.699|0.951|45.0|
|8.714|0.950|46.0|
|8.706|0.950|47.0|
|8.701|0.951|48.0|
|8.711|0.950|49.0|
