Project: Create a new zero-shot model with NLI data
DescriptionThe zero-shot classification pipeline has becomes very popular on Hugging Face. It allows you to classify a text in any category without having to fine-tune a model for the specific classification task you are interested in.The zero-shot pipeline is based on models trained on Natural Language Inference (NLI). This project will train a new NLI model, which can then be used in the zero-shot classification pipeline.Model(s)Any base-model can be used. Since there are already several NLI models on the model hub, I suggest to train a new model based on Microsoft’sDeBERTa-v3model.Version threewas only published few weeks ago and can outperform larger models (see an examplehere).We can probably create a new SOTA NLI model with the new DeBERTa version and enough NLI data.DatasetsEstablished NLI datasets include:MultiNLISNLIANLIOther interesting NLI datasets include:FEVER-NLIDocNLILingNLIMore datasets can be included!ChallengesNLI models can be trained as either 3-class classifiers (entailment/neutral/contradiction) or as 2-class classifiers (entailment/not_entailment). Both setups have different advantages and disadvantagesThere is a lot of NLI data (2 mio++ texts in the datasets linked above), which makes training computationally expensive. Optimising the training pipeline is a challenge.Many different datasets can be translated into NLI-format. Including more datasets can be beneficial, but requires manual transformation of datasets.Desired project outcomesCreate a Streamlit or Gradio app onSpacesthat provides an interface for zero-shot classification with a new NLI model in the backend.Additional resourcesSee the links to the datasets above. Also see Joe Davidson’s originalblog poston the zeroshot pipelineDiscord channelTo chat and organise with other people interested in this project, head over to ourDiscordand:Follow the instructions on the#join-coursechannelJoin the#zero-shotchannelJust make sure you comment here to indicate that you’ll be contributing to this project
2021-11-16T12:51:00Z
Hyee! I’d love to contribute in this one. I guess further discussion will take place in Discord?
2021-11-16T13:44:05Z
Hey@HarrySaini, yep Discord is probably the best place to coordinate / discuss more efficiently
2021-11-16T15:45:07Z
Interested. Messaged in Discord as well.
2021-11-18T17:35:34Z
I happen to have trained a bilingual MNLI model recently, so I decided to give this project ashotas well (without taking a place on the team). In theory this should work on English and Russian. Any feedback is very welcome.
2022-01-31T18:09:28Z
@MoritzLaurerwas there any update on this? I would be very interested in a superior model to facebook/bart-large-mnli
2022-01-31T18:23:42Z
yeah, this one is better than bart-large-mnli:MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli · Hugging Face(or this one:MoritzLaurer/DeBERTa-v3-base-mnli-fever-docnli-ling-2c · Hugging Face) and they should also be faster
2022-02-05T17:55:41Z
Great! Thanks, I will be comparing them for zero-shot text classification.
2023-04-03T08:35:54Z
Hi@MoritzLaurerIs there a way to update the first post with some example notebooks on how to take any transformer-based models and one or more NLI-based datasets and fine-tune a new Zero-Shot text classifier? (some architectures are missing from the Models Hub like BERT, I would like to add those)
2023-04-11T07:47:22Z
I used the code in therun_xnliscript and it worked well!github.comtransformers/examples/pytorch/text-classification at main ·...main/examples/pytorch/text-classification🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX. - transformers/examples/pytorch/text-classification at main · huggingface/transformers