HTML Embedding processing
I am interested in creating embedding to HTML tags in a web. Does Bert have such model?Other works that can be relevant
2022-01-06T13:40:10Z
Hi,MarkupLMis such a model. I’ll add it soon to HuggingFace Transformers.
2022-01-07T17:49:36Z
Can I already use it?
2022-01-08T09:09:57Z
Technically yes, as follows:!pip install -q https://github.com/NielsRogge/transformers.git@modeling_markuplm

from transformers import MarkupLMTokenizer, MarkupLMModel

tokenizer = MarkupLMTokenizer.from_pretrained("microsoft/markuplm-base")
model = MarkupLMModel.from_pretrained("microsoft/markuplm-base")

with open("path_to_html_file") as f:
       html_string = f.read()

encoding = tokenizer(html_string, return_tensors="pt")

# forward pass
outputs = model(**encoding)Note that there are still some bugs and rough edges, which will be fixed soon.
2022-01-22T21:52:18Z
Hi. Thanks so much for working this. I ran the code above in colab and am hitting the following error on the encoding line:_init_() missing 2 required positional arguments: 'merges_file' and 'tags_dict'Do I need to clone the unilm markup repo too somehow?
2022-01-23T20:22:26Z
I think what needed to be done is change:encoding = MarkupLMTokenizer(html_string, return_tensors="pt")toencoding = tokenizer(html_string, return_tensors="pt")
2022-01-24T08:41:33Z
Yes my mistake, thanks for fixing.
2022-02-12T18:20:50Z
@nielsrthanks again for working the markuplm, it was a good small learning experience trying to use your branch to get the thing running for fine tuning on the WEBsrc dataset per the creators’ git instructions. Draft space for qa on html strings is here:https://huggingface.co/spaces/FuriouslyAsleep/markupQAdemoI know it’s not your job but any idea why the fine tuning was gobbling up 70+ gb of ram before I trimmed the training set down to just one small single domain subset? Colab just kept giving out on me.
2022-02-13T19:13:47Z
Fine tuning colab is at:colab.research.google.comGoogle Colaboratory