Reproducible model between SetFit Versions?
Previously I trained a SetFit model using SetFit v0.6.0 (py 3.8)I have followed the migration steps and have refactored code to move to SetFit v1.0.3 (py 3.11).I have used the same hyperparameters, and the same random seeds and the same training data.Output of the v1.0.3 model is reproducible: the same model is created each time.Output of the v0.6.0 model is also reproducible.The two models are not the same, and their outputs are significantly different.Is this expected?
2024-10-15T05:17:42Z
I don’t use the NLP model much, but I think it’s normal for the output to be different between different versions of the AI model for more than a year…In a normal program, even a 10 year old code can work, but in AI, it can change in 6 months.But if they seem too different, maybe some default parameter has been changed or something. The output can change quite a bit depending not only on the version of the model, but also on the version of the library.github.comGitHub - huggingface/setfit: Efficient few-shot learning with Sentence...Efficient few-shot learning with Sentence Transformers
2024-10-18T01:55:20Z
@tomaarsen@lewtundo you have any insights?