Get the Q&A in LayoutLMv2 in text form
I am going through theLayoutLMv2 inference tutorialby@nielsr. This is a great tutorial to understand how LayoutLM is working.The final output of the model is in the form of bounding boxes on top of the invoice image. I would like to get the Question & Answer pair as the output (JSON if possible). My questions are -How can I get the Question and Answer text (instead of the bounding boxes)How do I cluster the words in Questions/ Answer (right now there can be multiple bounding boxes for a single Question/ AnswerAny help in this regard would be appreciated.
2022-02-07T20:17:50Z
Hi,We plan to add LayoutLMv2ForRelationExtraction (that allows you to do just that) to the library. Seehereto follow the progress (it also includes a link to a Colab notebook).