Understanding zero-shot classification in one-shot ;-)
Hello there!I LOVE the zero-shot classification API. When you think about it, it could also be used as search engine (find the documents that are related to a particular topic).My question is: conceptually, how does it work? I saw the original postNew pipeline for zero-shot text classificationbut I cannot find anything that explains the model under the hood.My understanding is thatthe API gets the embedding for the input sequencethe API computes the embedding for the possible labelsa similarity measure is computed between the input sequence and each of the labels and - via a softmax - a probability over the topics is then givenIs that correct?Thanks and keep up the good work!
2021-07-30T22:41:50Z
hey@olaffson, conceptually i believe the zero-shot classification pipeline works by adapting a task like natural language inference, where the language model is provided with a “template” like"<some text you want to classify>. This text is about <MASK>"and the model fills in the most probable label given the context. joe discusses this in this section of hisblog postand you can find one of the underlying zero-shot modelshere
2021-07-31T16:37:00Z
thanks@lewtun, I will have a look at the sources
2021-08-02T17:39:38Z
Glad you liked itI wrote a blog post about it last year, check outthis sectionfor a conceptual explanation.