Bitext Alignment (Translation Source and Target Alignment)
Hello!Looking for some guidance on handling Bitext Alignment. Similar to how Microsoft handles it with theirTranslation Service. For reference I’m using the Helsinki pre-trained models and have come across papers say alignment can be derived from the hidden states or the decoder attentions. I am returning them but can’t make any sense of the returned tensors. Looking for some documentation or examples on how to make sense of the decoder_attentions.generated = translation_model.generate(return_dict_in_generate = True, **prepare
translation_model = MarianMTModel.from_pretrained('Helsinki-NLP/opus-mt-en-es', return_dict=True, output_attentions=True, output_scores=True, output_hidden_states=True)
2021-03-10T14:04:44Z
If you want to extract a sentence representation (for alignment) out of the encoder output, you should consider using a pooling layer to obtain the sentence vector:sentence-transformers/Pooling.py at ec76488000f94efdba911356b8924cc46db0c2ee · UKPLab/sentence-transformers · GitHubyou can refer to the pooling class by sentence-transformers.
2021-03-10T19:38:26Z
Thank you@morenolqfor taking the time to respond! I’ll check this out.