Word_to_tokens() and word_ids() ---- microsoft/deberta-v2/v3
Hello,My code using the mapping functionsword_to_tokens()andword_ids()While usingdeberta-v2/v3tokenizers  I am getting the error msg:raise ValueError("word_to_tokens() is not available when using Python based tokenizers")
ValueError: word_to_tokens() is not available when using Python based tokenizers
python-BaseExceptionAny workaround for that?Thank in advance,Shon
2022-07-14T16:25:45Z
I am facing a similar issue with Flaubert’s Tokenizer where word_ids() Method is not working and I a shown the same error as you are! If you have found a workaround already; do a post with an update!
2022-07-14T16:37:49Z
No… I didn’t. Waiting for any help from here