Bert question answering model without context
Regarding question answering systems using BERT, I seem to mainly find this being used where a context is supplied. Does anyone have any information where this was used to create a generative language model where no context is available?
2021-03-25T14:28:27Z
Hey@EmuK, indeed most expositions of “question answering” are really referring to the simpler task of reading comprehensionWhat you’re probably looking for is either:open-domain question answering, where only the query is supplied at runtime and aretrieverfetches relevant documents (i.e. context) for areaderto extract answers from. You can find a really nice summary of these systems here:How to Build an Open-Domain Question Answering System?closed-book question answering, where large language models like T5 or GPT-3 have memorised some facts during pre-training and can generate an answer without explicit context (the “closed-book” part is an analogy with humans taking exams, where we’ve learnt something in advance and have to use our memory to answer questions). There’s a brief discussion of these models in the above blog post, but this T5 paper is well worth reading in it’s own right:[2002.08910] How Much Knowledge Can You Pack Into the Parameters of a Language Model?There’s also a nifty library called Haystack that brings a lot of these ideas together in a unified API:https://haystack.deepset.ai/
2021-03-25T15:45:32Z
Ok got it. Thanks for the references!
2021-08-04T10:44:58Z
Hi you should try RAG and RAG-end2end in the Transformers library.
2023-03-14T12:22:31Z
Thanks@lewtunfor your answer and references. I really have found the “haystack” helpful. And I am progressing with:LFQA - HaystackRAG - HaystackMy question isCan these models support the “structural data”? Meaning, can I give some sort of data with headings and the conventional records, and then able to query it? Is that possible with the given models? Or for that some specifics needed such as QA on Tables - Haystack?Please help out. Thanks!
2023-10-01T20:28:08Z
Hi Ayush Were you able to find out something related to this