pytorch t5 en license:mit region:us
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.01
- `lambda` = 0.02
- `alpha` = 1.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|11.994|0.936|1.0|
|11.877|0.941|2.0|
|11.963|0.939|3.0|
|11.879|0.941|4.0|
|11.783|0.945|5.0|
|11.723|0.946|6.0|
|11.880|0.941|7.0|
|11.739|0.946|8.0|
|11.663|0.948|9.0|
|11.671|0.948|10.0|
|11.658|0.948|11.0|
|11.637|0.949|12.0|
|11.687|0.948|13.0|
|11.685|0.948|14.0|
|11.804|0.944|15.0|
|11.721|0.947|16.0|
|11.607|0.950|17.0|
|11.713|0.947|18.0|
|11.683|0.948|19.0|
|11.658|0.949|20.0|
|11.636|0.949|21.0|
|11.700|0.947|22.0|
|11.601|0.950|23.0|
|11.821|0.944|24.0|
|11.587|0.951|25.0|
|11.623|0.950|26.0|
|11.640|0.949|27.0|
|11.627|0.950|28.0|
|11.621|0.950|29.0|
|11.651|0.949|30.0|
|11.597|0.951|31.0|
|11.650|0.949|32.0|
|11.651|0.949|33.0|
|11.642|0.949|34.0|
|11.637|0.949|35.0|
|11.593|0.951|36.0|
|11.615|0.950|37.0|
|11.594|0.951|38.0|
|11.646|0.949|39.0|
|11.631|0.950|40.0|
|11.635|0.949|41.0|
|11.605|0.950|42.0|
|11.629|0.949|43.0|
|11.638|0.949|44.0|
|11.586|0.951|45.0|
|11.591|0.951|46.0|
|11.609|0.950|47.0|
|11.607|0.950|48.0|
|11.604|0.950|49.0|
