Shockingly Incorrect Evaluate Function in Huggingface API
Two different accuracy, one from manual evaluation and one using huggingface’s trainer function. They are completely different. I am shockedScreen Shot 2023-11-03 at 3.05.20 AM1000×586 70 KB
2023-11-02T16:25:08Z
Sayan01:am shockedYou are callingdrop_lastin your eval dataloader, which in general you should never do. Does not doing so fix this?