Training a language model on Arabic data - handling right-to-left text direction
I am currently working on training a language model on Arabic data and I have encountered an issue with handling the right-to-left text direction. Specifically, I am unsure whether I should reverse the text in my training data to accommodate for the right-to-left direction, or if this would affect the quality of the resulting language model.As Arabic is a right-to-left language, the order of the words in a sentence is reversed compared to left-to-right languages like English. Therefore, I am concerned that reversing the text could change the meaning of the sentences and produce incorrect results. On the other hand, if I do not reverse the text, the language model may not be able to learn the correct relationships between the words and may not perform well on real-world data.I would appreciate any advice or insights on how to handle right-to-left text direction when training a language model on Arabic data. Specifically, I would like to know:Should I reverse the text in my training data or not?If I do not reverse the text, how can I ensure that the language model is able to understand the correct relationships between words and produce accurate results?Are there any other considerations or best practices I should be aware of when training a language model on Arabic data with right-to-left text direction?
2023-06-18T22:34:36Z
Yes, we need an answer to this very important question! If, during training, we pass is a RTL image/text pair to a ViT encoder-decoder (e.g. TrOCRmicrosoft/trocr-base-printed · Hugging Face), do we reverse the text description so the first character on the far right image becomes the first character on the far left text description?@nielsr- Any guidance please?Example in Hebrew:Text: שלום (<-- read RTL)Image:Does the encoder read the image LTR (in which case we should reverse out text description string)?Please help us folks working on Arabic, Urdu, Farsi, Hebrew models!
2023-06-19T10:56:09Z
@abdoelsayedCouple of clarifications needed:Are you building a brand new model of your own?ORDo you just want to train a model on Arabic text (without worrying about how it will handle the RTL problem), in which case have you tried using the tiiuae/falcon-40b model?Thanks.
2023-08-26T23:13:26Z
This will help:9.1. Working with Sequences — Dive into Deep Learning 1.0.3 documentation
2024-09-26T22:33:06Z
hello , so did you find a solution?